{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"76b63b99-8a0c-5eaa-9dc3-58e0633fe24f","task_key":"default--validation--5482","task_revision_id":"1","upstream_id":"5482","short_description":"HellaSwag validation 5482","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to clean dust mites [title] Steam clean carpets and furniture. [step] Dry vacuuming will not pick up dust mites from your carpets. Whenever possible, steam clean the carpets in your home.\",\"endings\":\"['First use an enzyme-resistant vacuuming cycle. Create a steam cleaning cycle using warm water and a mild dishwashing detergent.', 'The oil helps remove the larva, and the dust mites get picked up by hovering your vacuum over the cracks. [substeps] Start off by running your vacuum over your carpets, or by wiping over with a damp cloth once they are clear of dust mites.', 'The heat from the steam will kill the dust mites, making steam cleaning a highly effective method. [substeps] Steam clean your carpets and furniture at least 3 times per year.', 'Not only will this help to prevent dust mites, but your carpets will give you new ceilings. You may also opt to machine clean and air dry the carpets.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}