{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7358a975-665d-58b8-b06a-5cdc13cef0ac","task_key":"default--validation--2981","task_revision_id":"1","upstream_id":"2981","short_description":"HellaSwag validation 2981","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to take catnaps [title] Find a private location. [step] Most people prefer to nap in private since there could be many distractions in public places. If you're not home when you need to nap, consider napping in your office or in your car.\",\"endings\":\"[\\\"If someone unexpectedly arrives and hovers over you in a park or other public venue, you'll be less tempted to nap in that area. [substeps] If you have a confidentiality boundary or you're afraid of being overheard, ask your boss, a trusted friend, or coworker before taking the bubble bath that activated catnaps.\\\", \\\"If you don't have a car or office, ask if there's a private conference room you could use for a while. Leave a sign on the door stating that you shouldn't be disturbed.\\\", \\\"That way, you have somewhere to rest all day. [substeps] You can find many types of private locations on the internet like airbnb's or uber's.\\\", \\\"However, if you share a workplace with someone else or if you're working out at home, you could find an area that's conducive to napping where you won't be distracted by loud music. [substeps] If you have access to a microphone, use it to hear what you're doing from the car.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}