{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"421537f6-fcc7-56d3-839e-8c64be8b1d76","task_key":"default--validation--48945","task_revision_id":"1","upstream_id":"48945","short_description":"HellaSwag validation 48945","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to do an a + project [title] Begin with what the teacher assigns. [step] Your teacher should lay out the basics of what you need to do to create the project. Look at the description, and figure out exactly how many things you'll need to complete it.\",\"endings\":\"[\\\"Then, you're ready to start figuring it out in about one hour. [title] Figure out your assigned assignment the night before the test.\\\", \\\"But don't worry, this is done now! [title] Write down what you need to figure out. [step] There are plenty of methods for figuring out the answer to a question in the term, and it's helpful to have a starting point soon.\\\", 'You may be impressed by how easily you understand it, but if you know how diligently you can search for it, you could enjoy it. [title] Highlight what needs to be accomplished in each section.', '[substeps] For example, maybe your assigned project is \\\" pick a specific country to make a presentation on. You will need to represent this project visually.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}