{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a70921b0-027f-5f8f-b35f-20a9d3b88003","task_key":"default--validation--43254","task_revision_id":"1","upstream_id":"43254","short_description":"HellaSwag validation 43254","config":"default","split":"validation","body":"{\"activity_label\":\"Discus throw\",\"ctx\":\"The girl is back on the concrete circle swinging her disc and she then throws the disc. a black screen\",\"endings\":\"['appears with white powder on it and finally the words \\\" sky 45552010 \\\" on it.', 'appears again with red and white words that say \\\" other similar exercises on the same idea: focus on the low point of the implement trajectory.', 'appears with the scene split and a white text screen back in.', 'appears appears and it looks to be a website and some words appear and they read want to talk about them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}