{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"86c3e288-f081-5257-ae50-1683ffb9f007","task_key":"default--validation--30747","task_revision_id":"1","upstream_id":"30747","short_description":"HellaSwag validation 30747","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"A large orange machine is shown moving down a road with a claw like arm and razor cutting down the grass. the cutting\",\"endings\":\"['quickly continues as he also prepares to cut through national park.', 'continues and the machine is being operated in different areas such as highways.', 'then stops to talk to the woman who starts to stand and move closer to the machine.', 'is serious and it is for an intense time.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}