{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b7236998-c267-5ab5-903b-8dbbd967d4f1","task_key":"default--validation--44525","task_revision_id":"1","upstream_id":"44525","short_description":"HellaSwag validation 44525","config":"default","split":"validation","body":"{\"activity_label\":\"Futsal\",\"ctx\":\"A brazilian player pass the ball to his playmate who scores and runs to celebrate, other brazilian players scores many times. After, brasil plays with other teems and scores many times. then brasil\",\"endings\":\"['plays with players from other countries and scores.', 'scores again, they play more tennis and score a few times for the audience.', 'all coming back slowly, wins and gives a high five to his playmate.', 'reaches to give a high five with the brazilian player, joins a different brazilian player who scores even 109 and scores 108.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}