{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"a99b4183-b554-514f-965a-6687b7db5bce","task_key":"default--validation--1084","task_revision_id":"2","upstream_id":"1084","short_description":"HellaSwag validation 1084","config":"default","split":"validation","body":"{\"activity_label\":\"Removing ice from car\",\"ctx\":\"We see a man in a driveway cleaning snow from his car. the man\",\"endings\":\"['cleans the snow under his windshield wipers.', 'wet his car and wipes it down.', 'moves the snow out of the front bumper of his car and pours gasoline on it.', 'removes the pieces of the line off his windshield.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}