{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c8642b6-74ab-5292-851c-5eaf7d40d2e7","task_key":"default--validation--29060--variant-c29c0f1fbb27","task_revision_id":"1","upstream_id":"29060--variant-c29c0f1fbb27","short_description":"HellaSwag validation 29060","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to handle caffeine overdose [title] Call poison control. [step] This is especially important if you realize you've taken a medication high in caffeine or eaten or drank something high in caffeine. Foods that are high in caffeine include things like chocolate, and beverages like tea and coffee also tend to be high in caffeine.\",\"endings\":\"['[substeps] Make sure to place a call to poison control over the internet so you can access all of the information as quickly as possible. Also watch out for the belief that avoiding caffeine from your diet can trigger an overdose.', \\\"These are considered clinically low-grade behaviors. So if you suspect you're in the habit of eating junk food or other processed foods while taking otc medication, contact poison control.\\\", 'If you have symptoms like difficulty breathing, call poison control right away to figure out how to handle the problem. [substeps] The national poison control center (1-800-222-1222) in the united states can be reached at any time of day.', 'Focus your call on caffeine-free coffee (skim milk) and skim milk or tea. Add crushed nuts, hard candies, whole grains, or smoothies to your list of important foods.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}