{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2d37e4c7-34d4-53f7-b560-3cf9154a7f52","task_key":"default--validation--22525--variant-78931819e328","task_revision_id":"1","upstream_id":"22525--variant-78931819e328","short_description":"HellaSwag validation 22525","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to treat seasonal allergies in cats [title] Keep your cats inside when the pollen count is high in your area. [step] Visit an environment or weather related website to check the allergy forecast for your area. If the pollen count is high, you should keep your cat inside.\",\"endings\":\"['Avoiding contact with the cause of the allergy such as pollens, dust mites and molds, is the best treatment for cat allergies. [substeps] You could look up the allergy forecast on the weather network.', '[substeps] Pollen count regulations vary by region or city. In certain areas, allergy smoke indoors and outside may be more prevalent, but in others the pollen counts can drop and your neighborhood could become unsafe.', 'Invoicing a dosage into the local city or state laboratory is not recommended if you suspect allergies. [substeps] Windows are often turned off when pollen counts in the area are high, while indoor shops and parks tend to have low pollen counts.', 'If the pollen count is lower, keep your cat outside. [substeps] For example, you could check the pollen count by picking up your cat in the morning and putting it in your car, then placing the cat in a carrier or crate.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}