{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b0a06aa1-9079-5986-93a9-fe9f292b0b31","task_key":"default--validation--37444--variant-21f40bf0bc74","task_revision_id":"1","upstream_id":"37444--variant-21f40bf0bc74","short_description":"HellaSwag validation 37444","config":"default","split":"validation","body":"{\"activity_label\":\"Home,Categories\",\"ctx\":\"[header] How to call in sick when you just need a day off [title] Call your boss or supervisor early the next morning. [step] Don't delay--the sooner you tell your boss, the better. Additionally, you'll have a rather rough voice after waking up, giving you that added credibility.\",\"endings\":\"[\\\"Don't make calls when you've had a wonderful day. Even if you didn't finish it on time (try to be as friendly and upbeat as possible), do not feel as though you need to complain if it's been a good day.\\\", '[substeps] Even if your boss has an especially good day off, or you have something important to discuss, be sure to do it on a good day to avoid the call. [title] Set up a mobile phone plan with your coworkers if you need time off from work.', \\\"Plus, if you call early, you may be more likely to get your boss's voicemail or to catch your boss when he's off guard. If you call too late, then it'll look like you didn't really consider your boss's feelings.\\\", '[substeps] If at all possible, call earlier than you normally would. That way your boss may see your natural correctness, and you may pull off the prank better.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}