{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ecf9d8a3-2b25-5b3e-8853-49ef27f116ea","task_key":"default--validation--33303","task_revision_id":"1","upstream_id":"33303","short_description":"HellaSwag validation 33303","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to skip a class in middle school [title] Weigh the consequences. [step] Make sure the benefits will outweigh the consequences of skipping class. Know the rules at your middle school.\",\"endings\":\"['[substeps] Make your parents (and yourself) to talk about what to do if you missed a class in middle school, and help them stick to the rules. Perhaps other students also missed a class in middle school, and it may leave one with a bad reputation at one end, and a nasty reputation at the other.', 'Each school has a different absentee policy, but you will want to know what is going to happen if you skip a class before you decide to leave. Beyond the penalty dealt by your middle school, there are effects on your likelihood of success as a student if you are a habitual skipper.', 'Many students skip classes because they are confused about the world, and may not be able to put facts into their minds fully and quickly. Play it safe and always give students the benefit of truth and guess everything first.', \\\"Common rules for skipping your classes include that people who have blue skin are not allowed into the school and the wind may knock them out of class. [title] Compromise with your school's principal.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}