{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"62a0406c-91ca-5a64-acae-f36c8d8ff364","task_key":"default--validation--45176","task_revision_id":"1","upstream_id":"45176","short_description":"HellaSwag validation 45176","config":"default","split":"validation","body":"{\"activity_label\":\"Playing polo\",\"ctx\":\"An intro appears showing empty glasses, a card that say's \\\"audi polo challenge 2014\\\", an audi flag blowing in the wind, close up shots of a white horse, people with drinks, cars driving, polo riders getting on their horses, and a large group of people near and in a tent both standing and sitting. the polo match\",\"endings\":\"['is shown again, the winner of the event is a black horse, and the stands are filled with cheering fans.', 'begins and the men are riding on hroses that are going very fast as they all attempt to hit the ball with their sticks.', 'is shown again in slow motion, but the people are now sitting and riding off the field.', 'is a series of riders competing together and their opponents throwing balls around the racetrack.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}