{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9a1169f1-a42d-5c8c-aa32-b11f8918804b","task_key":"default--validation--31286--variant-ccb9a59a71c7","task_revision_id":"1","upstream_id":"31286--variant-ccb9a59a71c7","short_description":"HellaSwag validation 31286","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to transport cats by plane [title] Talk to the airline about transporting your cat in the cabin. [step] Contact the airline you are planning to fly with to check if you can transport your cat in the airport cabin in carrier under the seat in front of you. Avoid transporting your cat in the cargo or luggage hold if possible.\",\"endings\":\"['The airline may have instructions you should read before attempting to transport your cat in carrier. [substeps] Check that all the overhead fans are working.', \\\"Not only will it hurt your cat's health, but it will also cause your cat to turn infected and die. [substeps] Call the airline later and inquire about getting your cat in the cargo hold, particularly if it is a smaller carrier with less ground support and not providing additional shelter.\\\", '[title] Keep any belongings in the carrier secure until you arrive at your destination. [step] Avoid moving your belongings to the flight or to a hotel that offers in-flight amenities, such as cable car services or pet offices.', '[substeps] Most airlines will allow you to take your cat on the plane in the cabin area for a small fee. Try to call the airline well in advance of your flight, as there is a limited number of animals allowed in the cabin during a flight.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}