{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"db2ab655-7ed2-56f0-bcb2-4f33f737d5fd","task_key":"default--validation--41288","task_revision_id":"1","upstream_id":"41288","short_description":"HellaSwag validation 41288","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"The video leads into several clips of track runners jumping over a bar with their jump being shown again in slow motion. several more people\",\"endings\":\"['race through different obstacles as well as jumping over steep obstacles and their jump is shown again later.', 'are shown jumping over the bars in slow motion and ends with more clips of people jumping while jumping.', 'are shown jumping over the bar with one hitting the bar and not jumping over.', 'are seen jumping over the bars as well as beginning a slow race down the track, laughing with one another, and ending with the closing captions shown.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}