{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5584fc85-0fea-5eab-a71d-624a13c10a21","task_key":"default--validation--14263","task_revision_id":"1","upstream_id":"14263","short_description":"HellaSwag validation 14263","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to pass notes in class without the teacher spotting you [title] Make the notes as small as possible. [step] Smaller notes are easier to hide. Make small pieces of paper (about 3 by 5 inches/7.5 cm x 12. 5 cm) to pass in-between you and your friend.\",\"endings\":\"['Write things that your teacher might not notice. [substeps] For instance, write down spelling and grammar mistakes.', \\\"[substeps] If you're trying to pass notes by hand, make more small ones to be hidden. If your friend has a small notebook or pencil, write down how much of its contents they'll need to memorize.\\\", 'This will make passing between your friend and the teacher much easier. [title] Feel for the subject as you pass.', 'Perhaps make the little note papers outside of class and keep them ready stashed in your pencil case, to avoid having the teacher ask you to stop tearing paper during class. [title] Write the message on a spare scrap of paper, ideally when the teacher is writing on the board or talking to a classmate across the room.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}