{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e55d2942-e569-5886-861b-d68a176f2fa5","task_key":"default--validation--6864","task_revision_id":"1","upstream_id":"6864","short_description":"HellaSwag validation 6864","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"He walks throughout his backyard as he makes sure he covers the entire area as the rotating blades of the lawn mower cut the grass. after he\",\"endings\":\"['finishes, the yard is shown and begins to gargle of the compost in his yard.', 'pulls in the last string, he makes sure he has a good bend seated inside the weeds, grabs a rake then slivers the top of the hedge.', 'is done, he stands on top of a mound of new green fluff and begins throwing the fluff to the ground as he takes his cigarette out of his smoke holder and lights it.', \\\"'s done mowing, he plays with his two black and white dogs in his backyard.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}