{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"329f6f3f-a5bb-5a35-8391-06d730dce344","task_key":"default--validation--13593","task_revision_id":"1","upstream_id":"13593","short_description":"HellaSwag validation 13593","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to air dry oneself after showering [title] Wring out your hair. [step] If you have long hair, wring the water out of it with a twisting or squeezing motion. To do this, collect all your hair together and drape over one shoulder.\",\"endings\":\"['[title] Straighten your hair if it is straight or wavy. [step] If you hair has long hair, this can mean that it is wavy, not straight.', 'The other side of your head should hang over the edge of the shower or sink. This should not be left hanging down, or you will create unattractive, unsightly dampness.', 'Make sure that your ponytail is still securely in place. [substeps] Remember that hair will gather when dry without drying when wet.', 'Then grab your hair and twist it in order to help squeeze out any excess water and prevent dripping. [substeps] It is best to do this while still in the shower so that the water goes down the drain and does not make the bathroom floor wet and slippery.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}