{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9a2ec581-0510-5237-b16e-90400e19de71","task_key":"default--validation--45890","task_revision_id":"1","upstream_id":"45890","short_description":"HellaSwag validation 45890","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to deal with rats [title] Set out poison rat baits. [step] If you have an infrequent rat problem, or would rather not have to check traps on a daily (or weekly) basis, invest in poison rat bait. Simply set a small chunk of the poison bait in areas where you've seen rats, including the attic or basement of your home.\",\"endings\":\"[\\\"Avoid releasing it further back into the rat's home. See warnings below for making sure to have the bait out before, during and after feeding.\\\", 'Hold the rat securely up above your head, then turn on the floor lights or shut the window blinds. Drop a small pouch over your shoulder, and you have an easy kill for rats.', \\\"This will prevent rats from seeing the bait. You can also put the bait out during other times, like when you see rats on their way out of the house or when you're out at lunch.\\\", \\\"The poison is designed to smell appealing to rats, who will die once they've ingested the bait. [substeps] You can buy a bucket of poison rat baits at a local hardware store for about $25 usd.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}