{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7d38389-e945-5f0d-bc89-eb69cd3ad972","task_key":"default--validation--36406","task_revision_id":"1","upstream_id":"36406","short_description":"HellaSwag validation 36406","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to plan your honeymoon in bora bora [title] Plan a beach trip. [step] If you're honeymooning in bora bora, you'll want to pencil in lots of time at the beach. The most popular beach in bora bora is matira beach, where the water is crystal blue and the boardwalk is full of eateries, resorts, and souvenir shops.\",\"endings\":\"['Stop by a boardwalk and find a beach friendly bar or restaurant where you can lounge. [substeps] Choose as many chairs as you can afford to lounge in.', \\\"However, many wedding destinations don't have beaches in bora bora, so you're more likely to get a hawaiian vacationer near you. As with many tropical destinations, location determines what you want.\\\", \\\"Look into lesser-known beaches if you want to avoid traffic. [substeps] If you or your partner love aquatic life, plan a trip to bora bora's lagoonarium, a natural outdoor aquarium.\\\", \\\"However, there are also various beaches around bora regional and most of them are landscape-friendly and low-maintenance. Only plan a beach trip if you're already in bora bora.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}