{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9e0b6080-d58a-5d46-bdc4-1964808c72f5","task_key":"default--validation--1765","task_revision_id":"1","upstream_id":"1765","short_description":"HellaSwag validation 1765","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to get kids to behave in the supermarket [title] Be consistent with your behavior expectations. [step] Wherever you're going, it helps children to know how they're expected to behave, especially in public. Set some ground rules that you hold them to always, including at the supermarket.\",\"endings\":\"[\\\"This lets your children know that you're aware of their behavior and which you can enforce. [substeps] Be clear about which foods are good, which are bad, and which are not.\\\", \\\"The food you serve the kids to pick which food is best for their age group. Make the rules so your kids can think clearly about what goes in and what comes out, so they aren't distracted from their schedules.\\\", \\\"Be understanding and reasonably flexible, but don't allow bad behavior to become an exception or habit. [title] Inform your child where you will be going.\\\", \\\"[title] Label what food you're serving. [step] Children need to know what's allowed.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}