{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"23f32659-7982-5da1-bc00-91d57e3ecec0","task_key":"default--validation--12264","task_revision_id":"1","upstream_id":"12264","short_description":"HellaSwag validation 12264","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean flats [title] Wipe off the leather flats with a dry cloth. [step] Begin by using a clean, dry cloth to wipe off the leather shoes. Go over every part of the shoe.\",\"endings\":\"['Always wipe from the top of the foot up to the heel. Make sure to wipe to the sole below the heel.', \\\"Wipe off dirt and mud from the surface of the shoes. [substeps] Don't neglect the center part of the shoe as well.\\\", '[title] Wipe the sides with a leather brush. [step] Carefully sweep the entire surface of the sole, taking care to limit the scuff marks and scratches.', \\\"You don't have to scrub-just try to get off any loose dirt or debris. [title] Use a damp cloth to wipe once again.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}