{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5feacedb-8224-5097-afc0-d91dc72e3527","task_key":"default--validation--17609","task_revision_id":"1","upstream_id":"17609","short_description":"HellaSwag validation 17609","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to get bad smells out of a microwave [title] Place 4 teaspoons (20 ml) of white vinegar in a microwave-safe bowl with water and microwave on high for 6 minutes. [step] Leave the solution in the microwave to steam for an hour and then wipe the insides. Clean the microwave with soap and water.\",\"endings\":\"['[substeps] An alternative is to place a bowl of white vinegar in the microwave--without turning on the microwave--and allow the vinegar to sit there until the odor is gone. Then clean the microwave with soap and water.', 'Give the vinegar time to settle, but do not scrub. This method will not work if all you smell is the food.', '[substeps] If your microwave has a \\\" wipes \\\" function, you may not be able to clean it. [title] Add food flavorings (salt, basil, cinnamon, lemon).', '[substeps] Vinegar has a powerful residual smell if not removed quickly. With about 5 minutes of steam to a room or restaurant, vinegar is easily removed.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}