{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b43c514e-0f75-55ed-8c9b-f6fc3c522827","task_key":"default--validation--26291","task_revision_id":"1","upstream_id":"26291","short_description":"HellaSwag validation 26291","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to buy kitchen accessories [title] Know the model number of your appliance. [step] This can help you find the specific part (s) you need for your kitchen appliance (s). [substeps] Try looking inside a door of a major appliance or on the bottom of a counter-top appliance to find its model number.\",\"endings\":\"['For example, if you install a sink, see if its model number is written below the handle that opens and closes. You may be able to find the model number under the bottom side of the appliance (above the armpit).', 'You can also find information at www.iconic-technology hyperlink plus at the top-right of any door to find a model number. Enlarge your search criteria as necessary.', \\\"The number may be printed on an adhesive label. [title] Visit the manufacturer's website for your appliance.\\\", 'That is generally 1-8888. [title] Narrow down your own search to possibly some smaller appliance size.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}