{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a34e22e-81d0-5b93-9ba7-579162b19482","task_key":"default--validation--44187","task_revision_id":"1","upstream_id":"44187","short_description":"HellaSwag validation 44187","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to teach critical thinking [title] Start a class discussion by asking an open-ended question. [step] Open-ended questions are questions that have more than one answer. Make sure to emphasize to your students that there are no right or wrong answers.\",\"endings\":\"['Let them know that there are only opportunities to learn and some answers may work better than others. This will allow students to think critically and creatively without fear of getting anything wrong.', \\\"The more involved the group is in learning how to do critical thinking, the more prepared they'll be to answer answer your questions using correct answers. [substeps] For example, ask another class to discuss problems meeting certain criteria, depending on the circumstances.\\\", '[substeps] Open-ended questions include: \\\" what is the hour, time and place? \\\" \\\" what year are the oceans? \\\" and \\\" what does the sky look like? \\\" [title] Use action sentences to introduce the major ideas. [step] Prompt the students to think up important connections with the main ideas.', '[substeps] Students doing critical thinking will appreciate the question. About 90% of them will wonder about the correct answer, so they will be more receptive to show us our correct answer.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}