{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"421ea95d-c7a3-5939-ab7e-b338faaec050","task_key":"default--validation--1288","task_revision_id":"1","upstream_id":"1288","short_description":"HellaSwag validation 1288","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean tile with vinegar [title] Sweep or vacuum the floor. [step] Before cleaning a tile floor with vinegar, you will want to remove all debris from the surface of the floor. Use a broom, dust mop, or vacuum cleaner to remove all loose dirt and debris from the tile floor.\",\"endings\":\"['Once you are done, wipe up any excess dirt by brushing the floor with a soft bristled brush or nylon bristled duster. [substeps] Vinegar has long been used as a household product for its abrasive properties.', 'Ensure that you have a maximum of 60% vinegar stain from the cleaning cleaner and vacuum up any loose debris. Allow the stained area 24 hours to air dry and sweep and mop the floor.', '[substeps] Sweep the floor every week to ensure any lingering dirt remains on the floor. If you only want to clean the floor, sweeping the floor daily instead of once per week will take up significant amount of time.', \\\"[substeps] Vinegar is great for ceramic tile floors. Don't use vinegar on marble or granite.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}