{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d8f065cf-c879-56c1-9ca3-f06566ab3859","task_key":"default--validation--46490","task_revision_id":"1","upstream_id":"46490","short_description":"HellaSwag validation 46490","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to cross in front of a truck safely [title] If the truck sees you start to pass, continue passing. [step] Don't do anything the truck wouldn't expect you to do. [title] Observe the truck, the stoplights, and other lanes of traffic before crossing.\",\"endings\":\"[\\\"[title] Buckle your belt while entering a crosswalk lane. [title] Once you're in a crosswalk lane, stay for a few seconds.\\\", \\\"[title] Don't reduce your speed if the truck suddenly hebats you. [step] There are no limits to the speed your truck can control.\\\", \\\"[step] You might as well try blending in with the truck and stay safe in oncoming traffic. If you don't know them, try not to look at them to avoid eye contact.\\\", \\\"[title] Pass the truck as quickly as is safe. [step] To safely pass a truck, wait until you see the entirety of the truck's front in your rear-view mirror.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}