{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cbbeb37a-08d7-5560-a961-9619820209df","task_key":"default--validation--2568","task_revision_id":"1","upstream_id":"2568","short_description":"HellaSwag validation 2568","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to relax before an exciting event [title] Avoid caffeine. [step] Do not ingest any caffeine after 12:00 pm the day before your event. Caffeine increases your stress response and your perception of stress.\",\"endings\":\"['Even on stressful days you can still experience certain feelings, including anxiety and depression, about 1:30 pm, but be wary of taking along any caffeine. [title] Avoid smaller noisy and boisterous events.', \\\"This disruption could possibly prevent you from relaxing and feeling relaxed. [substeps] If you're very active, then trying to avoid caffeine might be easier, but do not do this unless you are fully engaged with your activity.\\\", 'If you are already feeling stressed about your event, drinking caffeine will make your body have a stronger response to stress. You may feel more relaxed if you do not drink or eat any caffeine (e.g.', \\\"Indulging in a non-caffeinated beverage can be particularly unpleasant and it's better to stop drinking than to develop negative reactions such as afternoon headaches. [substeps] Caffeine alone should not be a major cause of stress.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}