{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"615bdf08-8d84-5c6d-9b30-437a622eac8e","task_key":"default--validation--35232","task_revision_id":"1","upstream_id":"35232","short_description":"HellaSwag validation 35232","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to improve a toddler's speech [title] Get down to their level. [step] Squat or sit on the floor. This will get their attention.\",\"endings\":\"['You will talk with your child instead of at her. Also, she will be able to see your mouth and get visual cues on how to say certain sounds.', '[title] If needed raise hands together in fists. [step] If you stretch your hands to fists and do this while speaking, your child will most likely also do the same.', 'Try to just be as quiet as possible and only talk to them when they are calm. If you are talking for long enough, they will eventually hear your voice.', 'Let them ask you to move closer to them. If possible, use a head-high perch.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}