{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a4d310ce-e3cf-5ed5-a9e1-c8912c044adb","task_key":"default--validation--34704","task_revision_id":"1","upstream_id":"34704","short_description":"HellaSwag validation 34704","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to get a toddler to sleep alone [title] Talk to your child about sleeping alone. [step] Instead of making the switch suddenly, it is important to prepare the toddler well in advance. This can be done by talking to the toddler about changing where they will be sleeping and what it will be like to sleep alone.\",\"endings\":\"['If you feed them at night, light a dim candle or read a book with them in the dark, they will likely be somewhat disoriented. Having a quiet environment can help your toddler adjust to their new environment.', \\\"Your child will need to know that it's not too late, so they should agree to it. [substeps] Start by being forthcoming with your toddler about your child's needs.\\\", 'Consider talking to the toddler between meals. [substeps] When you eat dinner, you might tell the toddler that they will sleep in their bed.', '[substeps] For example, say, \\\" we feel like you\\\\'re ready for your big kid bed. Getting your big kid bed means that we feel like you\\\\'re ready to sleep all by yourself.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}