{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"22864df5-6517-5b8c-b113-a191ea594f10","task_key":"default--validation--27408","task_revision_id":"1","upstream_id":"27408","short_description":"HellaSwag validation 27408","config":"default","split":"validation","body":"{\"activity_label\":\"Using the rowing machine\",\"ctx\":\"Different angles of the rowing machine being used is shown. The woman change the resistance and the angle of the rowing machine. the woman\",\"endings\":\"['continues using the machine.', 'changes the schrind of the oar and the machine in between.', 'changes the stance and board to fit the whole of the machine comfortably.', 'change the push on the table using both feet.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}