{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4539283d-0975-5db3-a22e-9d3c75ac97f5","task_key":"default--validation--27541","task_revision_id":"1","upstream_id":"27541","short_description":"HellaSwag validation 27541","config":"default","split":"validation","body":"{\"activity_label\":\"Beach soccer\",\"ctx\":\"A group of young men and one woman play soccer in a sand pit using their feet to kick and sometimes their heads. the young people\",\"endings\":\"['struggle to keep up with the game as the woman try their best to score but fail by missing a single ball with her left hand.', 'a kicks the ball on the ground and then continuously kick it to the blocks while they struggle to get it in the actual game while they continue to kick and break the wall.', 'play in the sand to be give the video advance warning.', 'continue to play competitively sometimes slipping in the sand in an effort to dominate the soccer ball.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}