{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fbe1343d-7a96-5725-b5f1-33bcdf0c4daa","task_key":"default--validation--27807--variant-9539c9962255","task_revision_id":"1","upstream_id":"27807--variant-9539c9962255","short_description":"HellaSwag validation 27807","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to buy a stroller [title] Conduct research. [step] Search the websites of different stroller companies for testing videos to see how different stroller designs perform in a variety of environments, from navigating neighborhoods to boarding airplanes. [substeps] While you are online conducting research, consider checking parenting chat rooms for tips and stories from parents around the world.\",\"endings\":\"['Many parents have websites relating to the packages and models of stroller designs they bought their children with. Consulting organizations such as momoncab and scope resources such as search networks, jordans, and time sensitive consumer-friendly blogs can help you narrow down your search options.', \\\"This can give you some insight into stroller design. [title] Rate the stroller's current delivery time.\\\", '[title] Think about stroller rides and tests. [step] Watching young children try several of these alternatives will help you learn some basic aspects of the stroller and whether it will be the right fit for your unique needs.', 'If your child goes to daycare or preschool, ask the center if they have space to accommodate strollers. Some schools only have room for a certain type, such as an umbrella stroller.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}