{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"20d38c51-6831-5def-9077-fdcb8d31f378","task_key":"default--validation--22249","task_revision_id":"1","upstream_id":"22249","short_description":"HellaSwag validation 22249","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to tell the difference between prokaryotes and eukaryotes [title] Obtain a specimen slide. [step] Specimen slides of prokaryotes and eukaryotes can be obtained from biological supply companies. [substeps] If you are in school, ask your science teacher if they have access to slides.\",\"endings\":\"['[title] Place your specimen slide on the microscope stage (the platform that holds the slides). [step] Some microscopes will have stage clips that allow you to secure the slide in place to prevent movement during focusing and viewing.', 'During a science study or activity that occurs in the classroom, we will be asked what type of microscope, computer, cd/dvd or digital camera is used. If not, ask the librarian where there is one.', \\\"If they don't, they'll let you get one in a different, more requested process. The specimen slides will be engraved on chalkboard with measurements, with images, and in acrylic epoxy.\\\", 'If they do, you may not want to share the slide as you do not want someone to see your prokaryotes sample, but keep a journal of anything you find they share. If you go, you can carry the specimen (s) and record these onto another slide.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}