{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9af7d1be-92f6-5de6-9431-485e656877f1","task_key":"default--validation--13220","task_revision_id":"1","upstream_id":"13220","short_description":"HellaSwag validation 13220","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to exercise while watching tv [title] Position your machine safely. [step] Make sure that you can comfortably see the television while you're working out. If you have to crane your neck to watch tv, you need to move your machine.\",\"endings\":\"['Moving your machine too much or to too quickly will cause serious damage. Putting your machine down too much will cause your blood pressure to drop too quickly.', \\\"Keeping your neck at unnatural angles for long periods of time can cause neck injury. If you're watching tv on a laptop, elevate the laptop to shoulder height.\\\", 'Some models let you move around a bit in order to monitor the screen. Others are designed to catch you if you fall or do something that could potentially injure you.', \\\"It's recommended that you turn your machines on the lowest setting so that you can see the television as you sit. [substeps] You can be tempted to lean over the machine while you're working out, but that may only lead to injury.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}