{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4dd06585-6c7d-5be5-8d28-0cf81abdd8e9","task_key":"default--validation--44831","task_revision_id":"1","upstream_id":"44831","short_description":"HellaSwag validation 44831","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to create a spa kit [title] Get the right basket. [step] To start, you want the right basket for your spa kit. You do not want a relaxing spa day interrupted by fumbling to find items.\",\"endings\":\"[\\\"Look for a basket with multiple compartments so you can separate items by type. [substeps] Before choosing your basket, think about what you're gong to include in your spa kit.\\\", '[substeps] Be sure to choose a basket with a few compartments. For example, if your spa has a large pool, it will require a large basket.', \\\"Pick out a nice, safe basket for your spa kit. Make sure it's big enough to hold just the essentials you will need without being too big to fit much.\\\", 'Your chosen basket should contain : [substeps] Flowers: the type of flowers you desire. You can buy a variety of, so decide your own style.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}