{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"26e8aaf4-313f-5b7c-bb0b-051413a0a5f1","task_key":"default--validation--32403","task_revision_id":"1","upstream_id":"32403","short_description":"HellaSwag validation 32403","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to introduce your toddler to art [title] Let them explore materials. [step] With young children, art isn't about creating a beautiful final product. Let them learn about materials and suggest what they might use.\",\"endings\":\"[\\\"In your first years, be sure to start adding objects for your toddler to experiment with. As they learn, they'll be ready to handle new items.\\\", 'Before starting a project, show them what they will be using and allow them to explore their materials. Let them look at and touch items that engage their senses and open up their self-expression.', \\\"[substeps] Give materials to your toddler when they first start working on them, with vague directions indicating what they should and shouldn't use. Let your toddler see you doing the work in the garden, the park, parking, etc.\\\", 'Here are some examples of things to try : [substeps] Describe food handling techniques. A waiter works on hiding food in pots and pans.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}