{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8b83da84-8a9c-5402-a172-c0e3e0bed9fd","task_key":"default--validation--37754","task_revision_id":"1","upstream_id":"37754","short_description":"HellaSwag validation 37754","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to cross the canadian border with a dui [title] Use a temporary resident permit (trp) as a short-term solution. [step] The trp only allows you to enter canada for a specified reason, and only for a limited period of time. If you need to return to canada, you must apply for another trp.\",\"endings\":\"[\\\"However, if you are a current provincial resident, you must apply for a new student's permit. [substeps] Once both of you have become residents of canada, you can apply for a temporary student's permit.\\\", 'The trp is required for both curable and secure crossing within the canadian border. [substeps] A background check (anyway, the visa alone is not sufficient) is allowed to ensure that you are not carrying an exported foreign to canada.', 'When you go back to canada, you can download an form from https: // www.sos.state. Gov/voters/forms-district-bee-forms-archives-citizen-citizens-chambers /.', '[substeps] You cannot apply for rehabilitation until 5 years have passed since your conviction. During that 5-year period, if you need to travel to canada, a trp is your only option.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}