{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c3e31f6-56c9-5573-9e94-c3c3ed80f17c","task_key":"default--validation--47285","task_revision_id":"1","upstream_id":"47285","short_description":"HellaSwag validation 47285","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to attach a usb drive to your computer [title] Install any necessary drivers first. [step] If you still have the packaging for the usb drive, check the instructions to see if a driver is included for installation. A driver is a piece of software that allows your computer to communicate with the drive.\",\"endings\":\"['You can install an additional driver if needed. [title] Find the size of the usb drive.', 'All operating systems are designed to automatically download the required drivers, but if you have no internet connection, or if the instructions specify to install the driver manually, you should do so. [substeps] The driver will generally be included on a cd if manual installation is necessary.', 'If the driver cannot be installed, then your device will not receive or run a usb drive. [substeps] Check the health information manual if you do not have a driver already installed on your computer.', '[title] Make sure the data on the drive is appropriate for your computer. [step] If you want an updated drive system, you will need a memory card.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}