{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cba87b99-c90e-5525-bd43-db0fdc7c508d","task_key":"default--validation--10296","task_revision_id":"1","upstream_id":"10296","short_description":"HellaSwag validation 10296","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to be a purchasing agent [title] Secure your high school diploma or ged. [step] Take classes to complete your high school degree, if you have not done so already. If enrolling in school is not an option for you, you will need to take and pass the ged test, which is the professional equivalent to a high school diploma.\",\"endings\":\"['Use ged.com to find study materials and a test center near you. The test is only administered in person on certain dates, so study with a test date in mind.', 'Securing your high school diploma or ged requires you to take the private law practice examination (jd), which is an aptitude test that tests your proficiency in business skills. [substeps] Your high school diploma or ged cannot double or triple your coursework.', 'While taking the ged test is not necessary, it can help to get a good background on the property you are purchasing. [substeps] Some quality courses are offered through the national institute of realtors or the american society of realtors.', '[substeps] The test is offered in two levels: a semistandardized test that consists of two multiple-choice questions, and a verbal exam that consists of three combination questions. A verbal exam consists of 2 multiple-choice questions of 22 multiple-choice variables.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}