{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f5e4df8e-5e52-56be-a73f-a065214a614d","task_key":"default--validation--9737","task_revision_id":"1","upstream_id":"9737","short_description":"HellaSwag validation 9737","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"Suddenly, it is morning, and there are four men running in front of a truck filled with men as others watch. the men\",\"endings\":\"['then begin to fight one after the other.', 'begin to stuff the truck with boxes of clothing, which are being thrown on to the ground before they start to wrestle the wrestlers with blankets and rope.', 'are shown flying kites while a man hands out pamphlets for the crowd.', 'run into a building.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}