{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"75dae166-a041-5908-8fc9-09c45a2a2ccc","task_key":"default--validation--6573","task_revision_id":"1","upstream_id":"6573","short_description":"HellaSwag validation 6573","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to treat dermatitis [title] Recognize symptoms of contact dermatitis. [step] Contact dermatitis is one of the most common forms of dermatitis. This type of skin inflammation is caused by exposure to an allergen or irritant (such as poison ivy).\",\"endings\":\"['Signs of contact dermatitis include a localized rash, red patches and bumps, and localized itching. More severe symptoms include blisters and swelling.', 'Contact dermatitis ranges from mild itching to severe itching. All the skin reacts with different heat and kind of irritation.', \\\"Although there is no direct treatment, dermatitis can occur from just touching or handling a person's skin. Contact dermatitis occurs only on the skin of one or both arms, legs, hands, fingers, and feet, as well as the skin on the hands and feet.\\\", 'Contact dermatitis occurs when the skin surfaces in contact with fire, smoke, or other bodily irritants. This is a rash that looks like a blister.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}