{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b73538ec-1a63-5877-a761-4474bc56e57d","task_key":"default--validation--17960","task_revision_id":"1","upstream_id":"17960","short_description":"HellaSwag validation 17960","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to turn on do not disturb in ios [title] Manually turn on do not disturb. [step] In ios 7, swipe up from the bottom of the screen to bring up the control center. Tap the moon icon to toggle do not disturb.\",\"endings\":\"['[title] Tap your profile picture to open your profile. [step] You will be brought to your profile page.', 'Tap back up to view the upper portion of the screen. Tap the number next to \\\" stop \\\" to proceed.', 'Below \\\" settings \\\" you will see a selection of options. Tap those options to bring up the feature, allowing you to turn on turn off turn off turn on turn off settings.', 'In ios 6, open settings and toggle the do not disturb slider to on. [substeps] When do not disturb is enabled, you will not be notified about calls and alerts while your phone is locked.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}