{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0450877e-a95b-5a81-9838-53dec1c0827c","task_key":"default--validation--33470","task_revision_id":"1","upstream_id":"33470","short_description":"HellaSwag validation 33470","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to handle a snow day as a stay at home parent [title] Schedule play time with your child. [step] A snow day is a fun day to engage with your child, but it is important to manage your time so you can complete your own tasks. When you child is home unexpectedly, review your schedule for the day and consider where you can make adjustments.\",\"endings\":\"[\\\"[substeps] Push back non-essential household chores until the following day to free up some time to play with your child in the snow. Consider ordering take-out for dinner so you don't have to spend time cooking.\\\", \\\"[substeps] You should decide how much time you want to devote to each child. Each child is different, so be aware of your individual child's daily needs.\\\", \\\"[substeps] You may want to take a more active role by talking to your child's kindergarten teacher about the behavior of the child. See what you can do for your child and the best way to do it.\\\", '[substeps] The hotel room is usually an appropriate time for play. At a restaurant, for example, maybe you could get some ice cream in and let your child try to make scrambled pancakes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}