{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e5eef2e-4530-5deb-a2a4-64c4726a9549","task_key":"default--validation--37505","task_revision_id":"1","upstream_id":"37505","short_description":"HellaSwag validation 37505","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean ugg boots [title] Use a suede brush to comb the outside of the boots. [step] This gets rid of any large pieces of dirt which might be stuck to the outside of the boot, while also softening the nap and preparing the boot for the cleaning process. Suede brushes can be found at most good shoe stores.\",\"endings\":\"['[title] Coat your boots with a fleece or a suede rug to give them an instant soft finish. [title] Dampen a microfiber cloth with colored leather cleaner to get the results you want.', '[substeps] Start in the front, under the heel, and move around the top and bottoms of the boots. Move down to the calf, and then do the same here.', 'If you are in a hurry, be sure to bring your shoes home before the cleaning process begins. [title] Give the boots a thorough redress.', '[title] Brush the boots as described in the previous part. [title] Wet the outside surface of the boots.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}