{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4ad9ff8d-ccd2-526f-bf9c-e907a4043ada","task_key":"default--validation--12914","task_revision_id":"1","upstream_id":"12914","short_description":"HellaSwag validation 12914","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to transfer video [title] Connect your mobile device with your computer. [step] Connect your ios mobile device to your mac computer using the usb cable included with purchase. Itunes will open automatically once the device is connected.\",\"endings\":\"['[title] Select \\\" music services \\\" from the list of options. [step] You can change your audio track\\\\'s playback frequency, the new video stream available, and the downloads you plan to share with your media player.', '[substeps] Refer to the \\\" about \\\" page for instructions on how to connect your device to your computer. In order to connect your ios device to your computer, follow \\\" browse for my device \\\" at the bottom of the left menu.', 'If your device is viewable in the panel on the far left side, your device has been connected. [title] Select which type of media you wish to sync with your computer.', '[title] Sync all the software connected to your mobile device into itunes. [step] Make sure the connection is securely protected, in case the device disconnects.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}