{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c1e4cb11-5aa4-5466-ab15-c41b1eb6ccfe","task_key":"default--validation--8493","task_revision_id":"1","upstream_id":"8493","short_description":"HellaSwag validation 8493","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to treat congestion with herbs [title] Use eucalyptus. [step] Eucalyptus is typically available as an oil instead of an herb. However, it is also available in lozenges, cough medicines, rubs, and bath salves.\",\"endings\":\"['You can purchase the herb inexpensively from your local pharmacy or online. [substeps] Be sure to stock up on the herb you already use and one of its medicinal properties.', '[substeps] You can purchase it over-the-counter or you can make something with it. You can massage it into your sinuses with your fingertips or a cotton swab.', '[substeps] Eucalyptus makes digestion a natural part of normal life. It is commonly used to treat congestive heart failure.', 'It has a strong scent and decongestant properties that can help with mucus build up. Eucalyptus has been proven safe for pregnant and nursing women as well as children and it has the gras status in the us..']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}