{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2dd0f5f5-03c5-535d-89a7-76257c8d6f65","task_key":"default--validation--38763","task_revision_id":"1","upstream_id":"38763","short_description":"HellaSwag validation 38763","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to identify a russian blue [title] Learn about the breed. [step] The russian blue is a type of purebred or pedigree cat. The origins of the breed are unknown, but it is believed they originated in the archangel islands in north russia.\",\"endings\":\"['[substeps] This breed has a 20-20 fighting age and is well over 50, so it is very long-bred. It is a hardy russian cat with a robust brittle stripe.', 'The russian blue is distinguished by the \\\" train hounds \\\" approach. However, it is known to come in the form of a tight snowball, barbel.', 'Coloring, fur, and tracking work are different than purebred breeds with a wild color. The russian blue does not present itself in other ways, such as either male or female, often with the black cropped hair.', 'The weather there is extremely harsh in winter, which is why the russian blue developed such a thick, plush coat for insulation. [substeps] The breed started spreading to other parts of the world in the second half of the 19th century, and reached the us in the early 20th century.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}