{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f888a74e-d893-5214-9161-df75bbcb1ed6","task_key":"default--validation--48227","task_revision_id":"1","upstream_id":"48227","short_description":"HellaSwag validation 48227","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean straw hats [title] Make sure it can be cleaned. [step] Look to see if the hat has a label of some sort. You might be able to find this under the brim.\",\"endings\":\"[\\\"Make sure it won't stain or discolor and then pick the hat off. If it does, remove it and wipe with a cloth dampened with white vinegar.\\\", 'If the label says \\\" durable \\\" or \\\" in good condition, \\\" then it should be clean. If you are unable to clean your hat, consider trying something different.', 'If it doesn\\\\'t, you can know this person isn\\\\'t a regular worker at your company. [substeps] The label looks like \\\" gross hat cleaning.', 'Once you find it, see if your hat is made up of baku, shantung, or any other material that can be washed with water. It is possible to have a straw hat that cannot be hand-washed.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}