{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"244326db-0830-5e53-bca4-68d5bf58ad33","task_key":"default--validation--48588--variant-0a756df95392","task_revision_id":"1","upstream_id":"48588--variant-0a756df95392","short_description":"HellaSwag validation 48588","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to remove engine oil stains [title] Scrape off extra grease or oil. [step] Before acting to clean an oil stain, you need to remove any extra grease or oil. By removing extra grease or oil, you'll ensure that the stain does not spread and you'll make it easier for the cleaning products you use to fight the stain.\",\"endings\":\"['It will also make it easier for you to reshape and repair the stain. [substeps] Start at the top of the oil stain and scrape away any excess grease.', '[substeps] Most oil stains can be removed with hot soap or water, but you might notice an ingrained oil smell. You can use a paper towel or paper plate to try to work it off, but be careful not to create wet streaks.', \\\"[substeps] Take a razor blade or another thin object and slide it between the oil and the object you're cleaning. If you're cleaning a fabric, you may want to substitute the razor for a thin piece of plastic or a thin piece of cardboard.\\\", '[substeps] Instead of using scouring pads or food scraps, use a vinyl wire brush to lift off excess grease. [title] Use a stain remover formulated for grease.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}