{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"531b9b8c-b5ce-511f-9755-daa87551c369","task_key":"default--validation--22239","task_revision_id":"1","upstream_id":"22239","short_description":"HellaSwag validation 22239","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean raw denim [title] Avoid washing your denim for as long as possible. [step] Raw denim has not been pre-washed. Instead, you break it in and soften it with wear.\",\"endings\":\"[\\\"[substeps] Pre-washing doesn't clean off stains in your denim. Instead, you wash the denim until it is dry.\\\", 'The baking soda will thin out the fibers and fade them a bit. The less revealing it is, the better.', 'You have to continue doing this until it has been washed. [substeps] Even clothes with \\\" pre-washed \\\" labels can easily get moldy.', 'If you wash raw denim, the indigo dye bleeds out and it can alter the perfect fit you have broken in. Instead, wear your jeans as long as you can without washing them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}