{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d20b495a-de95-59f3-96eb-f1fe246c7a04","task_key":"default--validation--27211","task_revision_id":"1","upstream_id":"27211","short_description":"HellaSwag validation 27211","config":"default","split":"validation","body":"{\"activity_label\":\"Discus throw\",\"ctx\":\"A man with a blue shirt, blue sweat pants and green socks is standing outside in a large field walking into a circle of cement. once the guy\",\"endings\":\"['is inside, he extends a hand and begins hitting a wall.', 'is inside, the man begins spinning in circles and putting the pipe in his mouth.', 'is on the circle, he begins walking around the circle to find his perfect spot.', 'has done some stretching the man starts using and picking up metal scraps.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}