{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"621771fd-26b9-5c8e-9072-7a2168b196e6","task_key":"default--validation--43216","task_revision_id":"1","upstream_id":"43216","short_description":"HellaSwag validation 43216","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to travel overseas by oneself [title] Collect all related clues and local information, such as manners, culture ,. [step] .., etc. A \\\" planet \\\" travel book is recommended.\",\"endings\":\"['If you are in the us, then visit alice sakamesh 3 on the internet. If you are visiting canada you will be in europe and a historical history book related to the northern european region of australia is recommended.', '\\\" lonely planet \\\" publishes comprehensive travel books for almost every country and major city. In this guide, the traveler can read about the place\\\\'s culture, history, sights, recreation, hotels, restaurants, night life, camp sites, warnings, safety precautions, etc.', '[title] Keep a list of all your trips in there (this will help you stay on topic! [title] Take travel classes. [step] This will help you better understand several different areas of study.', '[title] Make a detailed itinerary of transportation between the countries you are visiting. [step] Include stop signs and other modern landmarks along the way.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}