{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9610b98a-9e9c-52ff-abbe-75c171712747","task_key":"default--validation--29551","task_revision_id":"1","upstream_id":"29551","short_description":"HellaSwag validation 29551","config":"default","split":"validation","body":"{\"activity_label\":\"Calf roping\",\"ctx\":\"A couple of men run out to the field and undo the ropes off the cow. two men on horse\",\"endings\":\"['carry the cow into a pen and the men tie her riding board.', \\\"run out to the calf's leg, untie his feet and get him back on the horse.\\\", 'go out to the field.', 'cross the ropes and try to fly the struggling cow to safety, but it jumps over them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}