{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c80e1cc6-ed14-52ba-9e37-cea2eeb6429c","task_key":"default--validation--5870","task_revision_id":"1","upstream_id":"5870","short_description":"HellaSwag validation 5870","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to react if a toddler swallows a coin [title] Remain calm. [step] It can be scary to learn your toddler swallowed a foreign object. However, the calmer you are, the better you'll be able to deal with the situation.\",\"endings\":\"[\\\"Don't panic-take a deep breath, count to five, and try not to give into your emotions. [substeps] Describe the experience calmly and explain what happened.\\\", 'Even if you think the child is choking, it is best to remain composed. [substeps] If you have the urge to call for help, do so.', 'Angry, upset, and anxious toddlers will often develop cravings for unknown foods or dangerous objects, which will often lead to eating difficult food (such as foods like candy) or unsafe foods. [substeps] Write down a list of the things your toddler eats and, if necessary, the people who eat them.', 'In the aftermath, take a few deep breaths and then proceed to care for your child. [substeps] You also want to remain calm to keep your toddler calm.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}