{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a81cb98d-603d-53b9-ac06-f0b9291c0604","task_key":"default--validation--19354","task_revision_id":"1","upstream_id":"19354","short_description":"HellaSwag validation 19354","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write a descriptive paragraph [title] Make sure to choose a meaningful person, place, or thing. [step] Before you begin your paragraph, you have to be sure to pick something that is worth describing. If you want to describe a character, make sure the person is interesting in some way.\",\"endings\":\"['For example, if you would like to describe the movie you watch, make sure to choose someone who to be important to you. This can be a relative or close friend, or an acquaintance.', \\\"If you don't, then the reader has no idea what it is about them that makes them interesting. [substeps] For example, a strong boyfriend, girlfriend, or partner probably wouldn't fall for an extrovert.\\\", \\\"If you want to describe an object, make sure it has some deeper meaning beyond itself so you have enough to write about. If you pick a place, make sure you can describe it in a unique way that can capture a reader's attention.\\\", \\\"For example, maybe you have interested a young child. However, if the boy is thin, it's okay to say that he's tall but still stands out, and that was why you drew him in.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}