{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"583522a8-b6dd-5dae-854d-8f268a0f2d77","task_key":"default--validation--48459","task_revision_id":"1","upstream_id":"48459","short_description":"HellaSwag validation 48459","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"A young man is outside attempting to do a highjump behind the london logo. A male man then stands in front of the mat and talks about the jump. another male\",\"endings\":\"[\\\"is then shown as he takes off and clears the bar of the high jump and there's an instant replay.\\\", 'appears in before he is cased six times attempting to same doing the same jump in different locations and taking short breaks.', 'joins in and the two jump from different heights, falling hundreds of times when landing.', 'joins in and prepares to get his jump in acting steadier and hitting the mat.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}