{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8237acad-b9c4-59d8-a469-74e888161eea","task_key":"default--validation--46359","task_revision_id":"1","upstream_id":"46359","short_description":"HellaSwag validation 46359","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to make a blt sandwich [title] Cook the bacon. [step] If there's one thing that's absolutely, positively , 100% crucial to a blt, it's bacon. Delicious, delectable bacon.\",\"endings\":\"[\\\"Turn a burner on your stove to medium heat and place a large or medium-sized frying pan on it. When the pan is hot (you'll know it's ready if water flicked on to immediately sizzles), add your bacon strips and let them begin to fry.\\\", \\\"[substeps] Incorporate a few egg whites into the pan; the more the egg whites break down, the flakier they'll be. For a lighter, light but full-fat version of the recommended bacon, however, you can use a toaster oven.\\\", \\\"By making the blt quick, however, you'll hold off on topping it after grilling. When you check the temperature, it should be no longer than 200 degrees fahrenheit (50 degrees celsius).\\\", \\\"You can toast and drip some butter on it if it appeals to you, but that can seem cheesy. Since you'll be using butter, not bread, both of those things have to be used for a blt sandwich.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}