{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"97be7116-b186-5a81-8311-c688f4b0f788","task_key":"default--validation--33641","task_revision_id":"1","upstream_id":"33641","short_description":"HellaSwag validation 33641","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to husk corn [title] Set up an easy-to-clean workspace. [step] Expect shucking corn to get a little messy. At the very least, have a bag handy to throw away the husk's leaves and silken strands as you work.\",\"endings\":\"[\\\"After you've cleaned the entire corn in a bag, switch it out for a clean, dry working space. [substeps] If you have a cooler at work and work at home, put the husks, leaves, and silks back into one cooler, but leave the foliage on one place for later.\\\", 'Since shucking can be difficult, avoid working in the kitchen or near a stove. Stay away from easy-to-clean workspaces like : [substeps] The kitchen.', 'To make life even easier, line a trashcan or similar container with a bag so you can work over that, or even inside of it. [substeps] Alternatively, you can also line your kitchen counter with paper or other compostable material if you want to compost the leaves and silk.', \\\"Perhaps you don't even need to cover it with paper. [substeps] If you do use paper, wad it up to punch out air from the bag before beginning to husk it.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}