{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2011dc81-3577-510c-a2c2-1feb0933d022","task_key":"default--validation--17740","task_revision_id":"1","upstream_id":"17740","short_description":"HellaSwag validation 17740","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"A series of still shots then display of the women, and some men, at various stages of the marathon running, breaking smiling and laughing, many while wearing the \\\"fab 50\\\" t-shirt. the last few still shots\",\"endings\":\"['are of the marathon riders, like they were on a surfboard, watching waves on the water before the race started.', 'are of women throwing their arms up in victory and taking one last posed, group shot.', 'include people close up floating on the back of wake boards in the water.', 'are shown of them relaxing in chairs and still shots of scenery shown.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}