{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e0195d7d-ba77-5c4d-af6a-46d06a66b31d","task_key":"default--validation--44591","task_revision_id":"1","upstream_id":"44591","short_description":"HellaSwag validation 44591","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to freeze tofu [title] Drain the tofu. [step] If the tofu is in a package, open it up and drain off all of the water. Don't spend too much time on the draining process.\",\"endings\":\"['You want to keep the tofu mostly whole. [title] Mix the tofu and cream cheese in a blender.', \\\"If you aren't incredibly worried about texture and you just want to keep the food fresh, then feel free to skip the pressing steps and freeze the tofu directly after you drain it. [title] Place the block of tofu onto an absorbent surface.\\\", \\\"[title] Place the tofu in a bin. [step] A large bin works best, since it's easier to see how much is in there.\\\", 'Tofu is tender, and draining it too soon may cause the moisture to become trapped inside, causing more moisture to drain into the nooks and crannies. [title] Check on the tofu per serving.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}