{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f374829d-c585-58c8-84be-fcd29b3509bd","task_key":"default--validation--38367","task_revision_id":"1","upstream_id":"38367","short_description":"HellaSwag validation 38367","config":"default","split":"validation","body":"{\"activity_label\":\"Croquet\",\"ctx\":\"A man walks towards the other side of the yard, hit the ball and the wall went to the other side of the yard, a man bend over and hit the ball with the stick, the ball roll and missed the rectangular hole. the man\",\"endings\":\"['hit the ball and other men kneel and put some cups in the hole.', 'hit the ball and the ball went to the wall.', 'runs to an empty area of vegetable patch, bend over again and hit the wall again, and the man darts to the left.', 'entered the hole and start to climb around on the cord and lifted up a log and threw it, then he walkmpt to the other side, got up and walked on his knees to the other side.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}