{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d8b01af5-2db2-524e-8ed1-b972b5631524","task_key":"default--validation--10703","task_revision_id":"1","upstream_id":"10703","short_description":"HellaSwag validation 10703","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to find a child in a shopping mall [title] Determine whether out if the child is actually lost. [step] Children have really playful natures and malls offer lots of opportunities to have some fun. Have a look around the immediate vicinity, calling the child's name.\",\"endings\":\"['He or she may just be around the corner playing with whatever they could get their hands on. Sometimes they may even hide from you for fun.', \\\"You can also let the child's parents take you, as this could make it more fun if they ask about the child with a smile on their face. [substeps] Also, find out if the child is living with a relative by calling his or her name, or if it is the child's mother.\\\", 'If you need help finding a child in a shopping mall, such as a school student asking for a seat, ask the child where the child walks in the mall. If your child appears to be in their room, the child might not be lost and wandering around.', 'See if they are already playing in the mall. [title] Find out if the child is a child.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}