{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"140bb008-4652-591c-8e7c-7c07c77ccad7","task_key":"default--validation--31574","task_revision_id":"1","upstream_id":"31574","short_description":"HellaSwag validation 31574","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to play in the dreamworld on candy crush [title] Move forward in reality. [step] Dreamworld levels are similar and based on the levels found on reality, which is what we call the main or basic game before dreamworld came. You have to move forward in reality in order to move forward in dreamworld.\",\"endings\":\"[\\\"Here's how to move forward in your world : [substeps] Walk forward, from where you first started-you can usually move forward in time at any time by walking. In the dream world, your body is in motion, while in other times, your body is moving side-to-side.\\\", '[substeps] Go on high speed chasing candies. Candies like pixie bombs will move to danger.', 'You cannot get to a level in dreamworld that you haven\\\\'t beaten in reality. [substeps] If you try to get into a level in dreamworld that you haven\\\\'t beaten in reality, you will be blocked by a message, \\\" next dream is locked.', 'Ghosts are farther forward in reality than they are in the game. Move forward or push yourself back and forth while doing your quest.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}