{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36dee164-f5cc-5d59-9a2c-60217b8934e2","task_key":"default--validation--27890","task_revision_id":"1","upstream_id":"27890","short_description":"HellaSwag validation 27890","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to be a radio board operator [title] Obtain your high school diploma. [step] You can learn many things during your high school years that will be beneficial to you as a radio board operator. Math classes help you calculate time measurements.\",\"endings\":\"['Math is particularly high as you are asked for, research, and professional work experience. Many medical schools have their websites on their standardized tests.', 'Earning a high school diploma can be a great way to show initiative in a field you feel you can do. [substeps] High school diploma courses may require the basic skills of a broadcasting operator.', \\\"You'll need to be certified as a certified or certified climber in order to work on land. Your high school grades will likely be affected by your high school talent since climber membership increases your likelihood of being hired as a certified climber.\\\", 'Computer classes help you become skilled with new computer programs and software. Speech classes prepare you to speak on air, if given the opportunity.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}