{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0017be93-0ce7-5ad0-8e55-2b86950ef10f","task_key":"default--validation--6926","task_revision_id":"1","upstream_id":"6926","short_description":"HellaSwag validation 6926","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to switch medicare from one state to another [title] Determine what type of medicare you have. [step] If you enrolled in medicare through social security, you likely have parts a and b. if you were automatically enrolled in medicare and took no other actions, you have parts a and b. [substeps] Under federal law, parts a and b are standardized nationwide. Your medicare will follow you automatically when you move to a different state.\",\"endings\":\"[\\\"In this situation, your state will only give you part a through default. However, if you have multiple states in which to apply, you'll need to make one decision at a time.\\\", 'All you have to do is change your address. [title] Notify social security of your change of address.', '[title] Use the \\\" if \\\" line on the form. [step] Write down the type of medicare you want and the portion b qualifies you for chapter 11 status, which replaces your dependent will.', \\\"In addition to being applicable if you need social security assistance and it meets your state's elective requirements, it also includes parts b. [title] Decide which state you're applying from.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}