{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c2b3cbc-f697-5fed-88ee-4472dafd4314","task_key":"default--validation--9779","task_revision_id":"1","upstream_id":"9779","short_description":"HellaSwag validation 9779","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"A student athlete appears, dressed in casual shorts and a red t-shirt, and he proceeds to do practice jumps. The first jump is poor, and he knocks off the bar during the jump. he\",\"endings\":\"['finishes, sits the bar, and gets up.', 'enjoys himself laying in the sand as he inches toward the end of the track.', 'returns, but the jump is pointless, he still has practice, and he has never done it.', 'clears the bar on the second jump.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}