{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c0e745f6-d2b2-5d5b-ba98-e149eb12af52","task_key":"default--validation--30465","task_revision_id":"1","upstream_id":"30465","short_description":"HellaSwag validation 30465","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to catch and look after wild mice [title] Buy a humane trap. [step] The tip traps work best. Put a bit of peanut butter at the end of the trap and set it out where ever you think the mice are likely to go.\",\"endings\":\"['Mice will smell the peanut butter and will bolt towards it, so keep it out of reach. This trap gives you a very tall chance of catching and following the mice, so bring a few box batteries along.', \\\"A longrange hunting rifle or whatever you have can be used as a lure. [substeps] If you can't find a living mouse, look for a mousetrap as as well.\\\", \\\"[title] Check your trap every day, and hopefully it will be closed! [step] When it is, get a big high sided container with a lid (this isn't your cage) and put your trap in with the mouse still inside. Open the trap door and wait until your mouse comes out.\\\", \\\"They will get more aggressive as they learn to associate the smell of peanut butter with movement. If it is your house, set the trap where you know mice can't find it.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}