{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"79f89420-0b54-5832-930e-af16c8040e12","task_key":"default--validation--19042","task_revision_id":"1","upstream_id":"19042","short_description":"HellaSwag validation 19042","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to throw a teenage ballet sleepover party [title] Invite all the girls in your class if you can. [step] If not, select the ones who are your friends. [substeps] Be aware that some classmates may feel left out.\",\"endings\":\"[\\\"[title] Set a definite date and time slot. [step] Pre-teenage and teenage girl's schedules tend to fill up at a rapid pace and if you don't set a definite time and date they might start making other plans! [title] Make the invitations or buy some.\\\", \\\"When this happens, don't berate anyone for it. It may just be your friends making things awkward for you.\\\", 'If this is the case, have them make plans together. [title] Choose a good time for the sleepover.', 'If they say something like, \\\" we told him/her not to do it, \\\" feel free to say no. The reason why a student might be left out is because you said something that he/she didn\\\\'t know about, such as, \\\" that was annoying! \\\" or, \\\" i could feel sad because he/she would keep doing that.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}