{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"456eabd6-4c4b-5119-b0d6-66013b2c8dd7","task_key":"default--validation--14242","task_revision_id":"1","upstream_id":"14242","short_description":"HellaSwag validation 14242","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to breed a sapphire dragon in dragonvale [title] Verify you're at level 16 or higher in dragonvale. [step] Your current level status is displayed at the top left corner of the dragonvale home screen. [substeps] Tap on \\\" goals \\\" at the dragonvale home screen and complete the list of featured goals if you're not at level 16.\",\"endings\":\"['Your current level status tells you your current level status while the weather app functions as an added feature. [title] Tap the \\\" + \\\" icon at the top right corner of the screen to place your desired goal.', '[title] Tap on the dragon\\\\'s name in the action box next to the \\\" get to know your achievement success \\\" heading. [step] This will result in a green conversation window.', 'Completing the required goals will help you advance toward level 16. [title] Tap on the breeding cave or epic breeding island, then tap on \\\" breed.', \\\"[title] Select your breeding partner if you'd like to breed the sapphire dragon. [step] Your dragon will be born or hatch upon your first breeding quest.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}