{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0c6e3ee5-233e-5fd5-bb70-2f7ac5024748","task_key":"default--validation--18729","task_revision_id":"1","upstream_id":"18729","short_description":"HellaSwag validation 18729","config":"default","split":"validation","body":"{\"activity_label\":\"Futsal\",\"ctx\":\"A group of men play soccer on an indoor playing field in front of an audience in surrounding bleachers. Two teams play soccer competitively with the team in the yellow uniform scoring a goal. further\",\"endings\":\"['teams compete with the different teams in the soccer field.', 'around the stands, the team win and celebrate.', ', players score three goals on the replay and the audience cheers and claps with delight.', 'in the game the yellow team earns another goal and shakes hands briefly with each other along with clapping their backs in celebratory response.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}