{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd4cd646-1f32-5acc-980b-2e3eb84b5856","task_key":"default--validation--37658","task_revision_id":"1","upstream_id":"37658","short_description":"HellaSwag validation 37658","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to sand drywall [title] Put down drop cloths. [step] When you sand drywall, you are going to get powder everywhere. If you've ever had a flour accident in the kitchen, then you know exactly how much mess fine powder can cause.\",\"endings\":\"['Remove drop cloths and place them on the counter while you sand. [title] Cover the floor with drop cloths.', 'Sprinkle drop cloths over the outside of doorways and a drop cloth over the floor to prevent dust from collecting there. Apply a thin layer of sealant between the drop cloths.', \\\"Because you're going to be mixing up tiny pieces of drywall, you don't want to ruin the fun of this project. Just make sure it's clean and dry before you begin.\\\", \\\"You don't want to be cleaning that up from all over the room and the rest of your house, so start by putting drop cloths on the floor. Then, you would have to sweep or vacuum for days to get all the dust up.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}