{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7755c5e2-cdf4-5d1b-a550-1447e88f290c","task_key":"default--validation--20887","task_revision_id":"1","upstream_id":"20887","short_description":"HellaSwag validation 20887","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to pass the igcse exam [title] Throw out previous thinking. [step] Forget about the classic' listen to your teacher, do your work, work consistently' strategy. Instead, think long term.\",\"endings\":\"['Instead of thinking, \\\" i will never get to pass ncse, \\\" tell yourself\\\\' what will happen if i fail the test. \\\\' ,\\\\' what will happen if i fail this test ,\\\\'? [substeps] This is the tricky part.', 'The more preoccupied you are about the future, the less serious the exam will be. [title] With any student, keep work to a minimum.', \\\"Encourage yourself to always think about your chances for passing the igcse exam. To knock out the newbie, practice' what if' earlier in the course.\\\", 'Relax, enjoy and have fun throughout the year. There is really no need to work your butt off for the internal examinations or try to impress your teachers because, in the igcses, \\\" only the boards matter! \\\".']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}