{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b0030bbb-9945-5a72-b69a-ab939d3d1a29","task_key":"default--validation--6457","task_revision_id":"1","upstream_id":"6457","short_description":"HellaSwag validation 6457","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to become a psychiatric nurse practitioner [title] Maintain a high gpa in high school. [step] To become a nurse, you'll need to get a college degree. Prepare yourself for college by maintaining your highest possible gpa in high school.\",\"endings\":\"['[substeps] If you plan to teach at a high school level, get a gpa about 100% in english, math, and science. If you intend to be a psychiatrist or a part-time nurse, you may want a class test that will include courses in good english and multiple choice questions.', 'Continuing your high school education and completing coursework will help keep your grades high and stay in top shape, which will help you look good as a nurse. If your gpa in high school falls below your desired gpa, contact your teacher or their administration team to inform them of your high gpa levels.', 'Have a strong college major, and pursue a bachelor of science in psychology. [substeps] A high gpa may increase your chances of being a doctor.', 'Take high level math and science courses, and take advanced classes when you can. [substeps] Complete 12th grade level biology, chemistry, and math.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}