{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ec43aea8-78d7-56d2-9f7b-933c07c3e55b","task_key":"default--validation--6286","task_revision_id":"1","upstream_id":"6286","short_description":"HellaSwag validation 6286","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to fish for squid [title] Choose a light, long fishing reel. [step] Because you want to feel even the slightest changes while you're fishing for squid, choose a light, long rod. You can use anything from a six to 20-pound line, but the lighter, the better.\",\"endings\":\"['[substeps] Choose a chain fishing net with a several. Use a steel bucket to hold the squid.', \\\"[substeps] You really don't have to invest in special equipment--almost any type of rod and reel will work for squid fishing. [title] Try a squid jig for bait.\\\", \\\"Squid can start at 0-40 degrees. You can also use light crayfish, though you can use any fillet that you have at any time that you're fishing for squid.\\\", \\\"You'll likely need to use more or less reels during the day, especially during hazardous weather. [substeps] Before you start fishing, consider increasing the resolution as the squid spawn.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}