{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a7f1fc34-e8dc-5dfa-aab8-86e9556baff4","task_key":"default--validation--9887","task_revision_id":"1","upstream_id":"9887","short_description":"HellaSwag validation 9887","config":"default","split":"validation","body":"{\"activity_label\":\"Cleaning shoes\",\"ctx\":\"The boy wipes down the sole of the tennis shoe using the wet wipe towel. The boy dries off the shoe using a towel. the boy\",\"endings\":\"['rinses the shoe inside the folded towel with water.', 'picks up a high top pair and wipes it off with the same wet towel.', 'tapes down the rear portion of the shoe.', 'shakes his foot with music and the camera follows the movements.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}