{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"969a4946-6b0f-5318-bc4a-a94bd4354858","task_key":"default--validation--6647","task_revision_id":"1","upstream_id":"6647","short_description":"HellaSwag validation 6647","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to sleep comfortably on a hot night [title] Stop exercising a few hours before bedtime and drink plenty of water. [step] When you exercise, you raise your body temperature and retain heat. Exercising several hours before you go to bed will give your body time to cool down.\",\"endings\":\"['[title] Keep legs and feet as cool as possible during the day. [step] This is often the case with people who live on hot days.', 'Stop when you are not overly tired and give yourself a break before going to bed. [substeps] [substeps] Do not be embarrassed by physical exertion.', '[substeps] You should also drink lots of water throughout the day to stay hydrated. You may also want to keep water by your bedside.', 'The best time for you to exercise is at night. By going to bed at a certain time and resting, you will either wake up feeling refreshed, or fall asleep feeling heavy.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}