{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b07639eb-c5ea-5f34-b5b1-6755d2266392","task_key":"default--validation--639","task_revision_id":"1","upstream_id":"639","short_description":"HellaSwag validation 639","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to become a meteorologist [title] Take the right courses in high school. [step] If you want to become a meteorologist, start preparing in high school. Take advanced math and science courses.\",\"endings\":\"['Being a meteorologist is good enough! Not only does that give you skills to put in, it is also beneficial to go the extra mile. [substeps] Many meteorology courses are in english, chemistry, and math.', \\\"You might think you'll want to go into intermediate first, such as biology, but as the aries, you'll need to learn other subjects for your career. Take elective courses while still in high school.\\\", 'Many schools offer ap courses that can transfer into college credit. [substeps] Take calculus, physics, chemistry, and earth science.', 'Most meteorlogists are required to take at least one semester of high school in order to qualify as a meteorologist. [substeps] Getting this degree is just as important as getting your college education.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}