{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4faacc51-e63b-5e00-b287-ff5bb49d4eb6","task_key":"default--validation--33435--variant-3d4a258fccb7","task_revision_id":"1","upstream_id":"33435--variant-3d4a258fccb7","short_description":"HellaSwag validation 33435","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to enjoy palm beach with the kids [title] Consider what type of accommodations would best fit your family. [substeps] Look for pet friendly hotels if you will be bringing the family pet along. There are a number of resorts and spas that allow for pets to stay with your family for a fee.\",\"endings\":\"['You could ask a tour guide on either trip. Having a pet that is a pet, like a dog or cat, in your care will allow you to spend quality time with your pet and keep them entertained, while allowing you to have other pet enthusiasts along.', \\\"If your family has more than one pet you may choose to stay at different places each day. Your child's experience should help guide you towards choosing the most environmentally friendly hotel.\\\", 'Drive your rv or even rent one to make the trip to palm beach. There are award-winning rv parks within walking distance of the beach.', 'Make sure your hotel offers a secure environment and is clean and dry so the pets feel safe. Some of the most popular places are: costco international, sanctuary city, chile & calgary, natives international, san francisco international and albania international.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}