{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"97d54841-35cb-5c44-83c3-73477f759f10","task_key":"default--validation--32717","task_revision_id":"1","upstream_id":"32717","short_description":"HellaSwag validation 32717","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean asphalt [title] Remove loose residue from the asphalt surface. [step] Use the oblong push broom to sweep away leaves, grass cuttings, loose dirt and other residue that is lightly resting on the surface. Removing all loose dirt and grime will make the task of actually cleaning the asphalt much easier.\",\"endings\":\"[\\\"[substeps] Don't forget to throw away any leaf litter. When you are cleaning asphalt, you'll need to put it in a plastic bag to protect it from getting snagged on any dirt, grass and other debris that is usually on the asphalt surface.\\\", '[title] Soak the asphalt with a steady stream of water. [step] Use the garden hose to apply the water over the surface.', 'You may need to use a hose or a rake for this. Do not use a brush or any motor as this does not work as well or as efficiently as a brush or rake.', \\\"[substeps] Be sure to be as precise and thorough as possible so you don't scratch the surface of the asphalt. [title] Use a mild solvent like hydrometer to wipe off any crusted-on flakes and debris.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}