{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02f84c1b-6c05-52e1-bb62-22af7e1f9eb2","task_key":"default--validation--45637","task_revision_id":"1","upstream_id":"45637","short_description":"HellaSwag validation 45637","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to spend time on the train [title] If you're looking at a long, tedious train journey, read on. [step] Take up your seat and put all your belongings in the correct storage area (eg under your seat). You can call or text any friends or waiting relatives till the other passengers arrive and the train finally departs.\",\"endings\":\"[\\\"This will cut down some pre-travel boredom. [title] Now the train's pulled out of the station.\\\", 'Here are some good options for doing this : [title] Take travel photos. [step] Take some pictures where you can take memories of your journey.', \\\"[substeps] You can even arrange for extra seats by using various grab bars throughout the station. If you're climbing up stairs, use about 75 percent extra seat bars every five minutes.\\\", \\\"[title] If you're an experienced passenger, check out old magazines and listen to podcasts on your train ride home. [step] The sound of a train engine shaking, of course, is sure to scare you out of your seat and make it colder on the ride.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}