{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"412d9ea8-4913-5807-aa5a-f5f0d62644f3","task_key":"default--validation--45178","task_revision_id":"1","upstream_id":"45178","short_description":"HellaSwag validation 45178","config":"default","split":"validation","body":"{\"activity_label\":\"Playing polo\",\"ctx\":\"There are a lot of celebrities and even royalty at this polo event and every now and then they are shown, and sometimes they are also interviewed. between\",\"endings\":\"['the interviews the competing polo fans are seen talking with women and still images of them playing polo are shown.', 'matches the people all go out into the grass and put the patches of grass back into the holes that are missing them, and the polo game continues for another round.', \\\"interviews, the host appears in front of the table and sometimes she clips, then she doesn't do a thing, she's holding her composure to keep it together.\\\", \\\"interviews, there's a photographer there and he puts in an offer and a recording of the event that he can use.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}