{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c667a243-5872-57f8-9916-fa2c6a039cab","task_key":"default--validation--50102","task_revision_id":"1","upstream_id":"50102","short_description":"HellaSwag validation 50102","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to welcome a foster child [title] Introduce yourself by the name you would like the child to use. [step] Your welcome begins as soon as the caseworker walks into the home with the foster child. Use a soft, friendly voice to greet the caseworker and the child.\",\"endings\":\"['Follow up your introduction with a question such as, \\\" how are you today? \\\" followed by a warm greeting like, \\\" hi, i\\\\'m molly. \\\" once the child\\\\'s name is said, you can say, \\\" hello, i\\\\'m jeffrey and i brought the foster child here today to welcome them.', 'Introduce yourself by the name you would like the child to use when referring to you. Do not insist the child call you \\\" mom \\\" or \\\" dad, \\\" as this will be confusing to the child.', '[substeps] Say, \\\" hello! You are the child\\\\'s grangiver, \\\" or, \\\" this is jake. \\\" look at the name the caseworker gave the child and check for a sign indicating that it is wrong.', '[substeps] You can say \\\" dear, it\\\\'s nice to meet you mr. sims! \\\" or \\\" hello joe, you\\\\'re sure, \\\" where appropriate.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}