{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d5744c9-8db3-517c-bb47-d6a06b867c48","task_key":"default--validation--15764","task_revision_id":"1","upstream_id":"15764","short_description":"HellaSwag validation 15764","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to color your converse [title] Start with a clean shoe. [step] The best converse to work on are brand new. If you can't get new ones, you will need to clean the ones you already own.\",\"endings\":\"['That means you need to have a pair of converse shoes, so clean the shoes thoroughly. Then you can select any of the color combinations that were added to your converse to dye them.', 'Clean them up by washing them in a sink full of soapy water and trying to dry them. Run the shoes under the faucet.', '[substeps] Wash your converse with soap and water. Use cold water as if taking a walk.', 'This will help the ink stick and show up better. Wipe down the rubber parts with a cotton ball soaked in rubbing alcohol.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}