{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9e1e1420-01f8-5691-8882-cdb17ff0d7ac","task_key":"default--validation--19860","task_revision_id":"1","upstream_id":"19860","short_description":"HellaSwag validation 19860","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to backpack in australia [title] Be prepared for the sheer size of australia when travelling through it. [step] Australia is not like europe or north america where a town pops up every few tens of miles/kilometres. Australia is vast and there are are a great many places that are nowhere near any other places.\",\"endings\":\"['If you want to secure a heat-filled travel backpack with extra padding, do some research to see which places in the country provide good light. [title] If you want to have shelter by a car of any sort, research places which exist on average compared to the rest of the country.', '[title] Match your weight to where you need to travel. [step] Not only do kangaroo lifestyle demands necessities without the mane and tail, kangaroo diet demands essentials without the hair and fins, though.', '[title] Know that you will need a solid, secrud backpack. [step] A laptop, a recorder, a digital camera and a notebook are all reliable.', 'Most of the population centres hug the seaboard (especially the east). This means that you will need to consider where you want to backpack and how you will do it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}