{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"271694f8-2909-569e-abb9-da1876366d87","task_key":"default--validation--45179","task_revision_id":"1","upstream_id":"45179","short_description":"HellaSwag validation 45179","config":"default","split":"validation","body":"{\"activity_label\":\"Playing polo\",\"ctx\":\"Between matches the people all go out into the grass and put the patches of grass back into the holes that are missing them, and the polo game continues for another round. the polo match ends and prince harry and prince william\",\"endings\":\"['and stuffed guests walk off the field, and the young men from the polo team talk to people who played the game, and laugh.', 'return to their machine.', 'are seen separately receiving a white box from a man, while they talk and shake hands as people clap and take pictures.', 'hit the ball back and forth with their fans, and the players have a great time celebrating.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}