{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b634926d-42db-5c1d-b6d5-4e015e72ee3b","task_key":"default--validation--12247","task_revision_id":"1","upstream_id":"12247","short_description":"HellaSwag validation 12247","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to make a birth keepsake [title] Write a letter to your unborn child. [step] Write what you are feeling, hoping for them in the future, about your feelings of excitement. [title] Write another letter to your child the day they are born.\",\"endings\":\"['[step] Let them know what was going on that day. What events were taking place while you were awaiting their arrival.', \\\"[step] It may be a lot more time than you think it will take, so be patient. Some mothers find that their anger levels quickly skyrocket because they have always been waiting to hear the baby's first heartbeat.\\\", '[step] Do not skip any steps! [title] Get a large box. [step] Put seeds, crayons, and anything else you like in there.', '[step] It might be something as simple as \\\" i was hoping for them, and i wish they were here, \\\" or \\\" i wanted to be there for them and watch them grow up. \\\" [title] Put on some music on and sing songs to accompany your letters.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}