{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd76a1e1-f4db-591e-b900-475dc7ef5693","task_key":"default--validation--36448","task_revision_id":"1","upstream_id":"36448","short_description":"HellaSwag validation 36448","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to evaluate students [title] Define a specific learning objective for each unit, test, and project. [step] Create a concise objective that summarizes the subject matter, skills exercised, and learning goals. Define the skills students will develop with measurable, actionable verbs.\",\"endings\":\"['[substeps] Vary the values the tests and professors define, skills students can retain, or behaviors students cannot foresee. It may help to write down each of the assessments separately and set aside an extra-critical list of typical qualities students will be assessed.', 'Include details that tailor the objective to the subject matter of a specific unit or assignment. [substeps] A strong objective for a unit would be, \\\" students will identify and describe the key players and events of the second world war\\\\'s european and pacific theaters.', '[substeps] Write \\\" consider what constitutes a measurable goal. \\\" this is a specific way to highlight the evaluation objective.', '[substeps] Research on specific requirements is good-find out what students have in common with the unit and what their preferred answer is. Avoid using either a numerical answer or a formula.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}