{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2996ed7d-f203-53ae-be42-03307ef9e1a0","task_key":"default--validation--10122","task_revision_id":"1","upstream_id":"10122","short_description":"HellaSwag validation 10122","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to avoid losing socks in the washing machine [title] Keep pairs matching through the washing process. [step] Use safety pins, clothespins, or just roll each sock pair together to keep them from drifting apart. Secure socks to their matches from the moment you take them off until they're out of the dryer.\",\"endings\":\"['Re-insert the pins or clothespins once the sock pairs up. [substeps] If the clothing is a linen or cotton wear, you can also skip this step.', 'At that point, take the clip or pin off and roll them into each other. [substeps] Keep a safety pin on one of the socks while you wear it so that later you can quickly pin it to its pair and throw it in the dirty laundry with ease.', '[substeps] Clothes that match may wash as you load them, so be sure not to throw them away in the wash. Fold only the tops of pairs of underwear.', 'You can either use 1 sock pair or 1 pair of common ones. [title] Clean all your shoes before you put them on.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}