{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2768b236-f0d5-5685-abd9-682e0998b0f3","task_key":"default--validation--4566","task_revision_id":"1","upstream_id":"4566","short_description":"HellaSwag validation 4566","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to decide if you should hold your child back from kindergarten [title] Notice how they follow directions. [step] Kindergarten children need to be able to follow 2-3 directions at a time. For example, their teacher may say, \\\" take some buttons and glue them onto the paper.\",\"endings\":\"['\\\" give your child some 2-step directions and see how they do. They should be able to complete the task with little or no difficulty.', '\\\" to assure the child, they might say, \\\" tape them down. \\\" just don\\\\'t let them put down buttons on the paper.', 'Be sure that the button works properly before you put the paper on. \\\" the children should be able to try to change directions at different times.', '\\\" or they may say, \\\" take some ribbons and glue them onto the paper. \\\" [substeps] If your child has practiced following directions, then they should follow instructions to ensure you\\\\'re properly doing what they instruct.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}