{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0de71431-ada6-5fe3-84c2-87d9c8b19379","task_key":"default--validation--33438","task_revision_id":"1","upstream_id":"33438","short_description":"HellaSwag validation 33438","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to breed a river dragon in dragonvale [title] Verify you've reached level 14 or higher in dragonvale. [step] The river dragon can only be bred after reaching level 14. Your current level status is displayed at the top left corner of the dragonvale home screen.\",\"endings\":\"[\\\"[substeps] If you don't reach level 14, wait two days for the server window to be filled up with your current level. Wait three more days for the server window to be filled up with your current level.\\\", '[substeps] If you\\\\'re not at level 14, tap on \\\" goals \\\" at the home screen and complete the list of featured goals to begin advancing toward level 14. [title] Tap on the breeding cave or epic breeding island, then tap on \\\" breed.', '[substeps] The current level refers to your current level of current levels based on the dragon you want to breed. Note that this level is currently 4 level 13, which means you can still breed a river dragon, but you cannot breed one.', '[substeps] You can choose to breed a river dragon from any bird world, but this will only allow you to breed the river dragon in dragonvale. [title] Keep breeding river dragons for up to five cycles until you reach level 13.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}