{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"19d4d6a8-3ca4-5cde-a6bf-510a398fbf20","task_key":"default--validation--19706","task_revision_id":"1","upstream_id":"19706","short_description":"HellaSwag validation 19706","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to order a free watchtower bible [title] Fill out the bible study request form. [step] To complete this form, you will need some information about yourself. You will need both background information (your name, gender, and preferred language) and contact information (your nation, phone number, and address).\",\"endings\":\"['[substeps] To apply for a free watchtower bible study, you will need to download the link to the form for the login page at. [title] Review the required step-by-step instructions that came with the license agreement.', '[substeps] Depending on your country, you may also need to fill out a religious preference form. You may order a religious name change form if you live on the east coast, or you may order one if you live within the pacific rim.', 'Don\\\\'t forget to include the best time for the jehovah\\\\'s witnesses to contact you. Click the \\\" please contact me \\\" button when you\\\\'re finished filling out the requisite info.', 'To start the form, you will need to provide your nationality, place of origin (=), and the first five digits of your license plate number (=). This process may take a minimum of three days.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}