{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"28dbd439-f2bb-5d68-ba06-49991ca6541d","task_key":"default--validation--9485","task_revision_id":"1","upstream_id":"9485","short_description":"HellaSwag validation 9485","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to treat poison sumac [title] Take a cool shower immediately. [step] Urushiol oil takes about 30 minutes to enter the skin. As soon as possible after you've come into contact with the plant, take a shower with soap and cool water.\",\"endings\":\"['This soaks up the excess moisture and makes your skin look fluffier. [title] Wear some sunscreen with spf 15 or higher.', \\\"Then, unplug the plant and wash your hands vigorously. [substeps] If you're a beginner to using soap, it might be beneficial to take a test kit ahead of time to see if you have the proper conditions or microbes to use on your plant.\\\", 'This will help to eliminate the bacteria that settle in the sumac seeds. [title] Spread the sumac oil directly on the affected skin.', 'Even if you miss the 30-minute window, showering will remove the oil from your skin and prevent the poison sumac from spreading. [substeps] Avoid hot water, which will increase irritation.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}