{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"52f45265-0d6e-5a32-b260-086abe4a5911","task_key":"default--validation--28934","task_revision_id":"1","upstream_id":"28934","short_description":"HellaSwag validation 28934","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to save on prescription drug costs [title] Know your options. [step] The pharmaceutical industry spends loads of money to market brand-name drugs. This includes television commercials targeted at people like you, but also outreach to doctors like your gp.\",\"endings\":\"['[title] Buy simple, cheap versions of drugs. [step] Experiment with what your doctor recommends to find the best deal.', \\\"Some things to go for are social media accounts, google and myspace accounts, allowing you to watch and/or comment with your friends' friends. Social media accounts are better choices for people that have subscriptions and thousands of social media pages and include a wide variety of marketing, advertising, promotion and distribution.\\\", \\\"Often, the result is pressure to go with a brand-name, even when it is not cost-effective or the best choice. [substeps] Don't insist on a brand-name.\\\", 'Look into the following options, and be honest with yourself about which one appeals to you : [substeps] Inhaler that has been shown to aid with lower back pain, pancreatic motility, abdominal pain, and abdominal discomfort. Inhalers that are specifically targeted to push out the acid in your stomach and help limit high blood pressure.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}