{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1edb96e2-ec29-57f2-b582-3b69537a001d","task_key":"default--validation--36935","task_revision_id":"1","upstream_id":"36935","short_description":"HellaSwag validation 36935","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to become a baseball scout [title] Watch baseball with a scouts eye. [step] First, refresh your knowledge on the game of baseball if necessary. You need to have a strong understanding of the strategies and mechanics of the game.\",\"endings\":\"[\\\"If you don't, you won't be able to become a scout. [substeps] In addition to avoiding learning many of the tactics used in the game, you need to learn the plays.\\\", '[substeps] To combat your interest in baseball, maintain a constant stream of information on cricket, golf, tennis, and other sports like basketball and soccer. Get access to information through catcher information, which the batting committee, the nationals, and other league coaches may have access to while the scout is away.', 'It is valuable for an associate scout to know as much as possible about playing the game of baseball so he can relate to prospects. [substeps] Baseball is in some ways a very technical sport, which means that baseball scouts need to have a keen eye and meticulous attention to detail.', 'This will help you keep in top shape and better compete against opponents of the same rank, race, or disposition. Be sure to familiarize yourself with all of the fundamentals prior to studying the game.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}