{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02f615aa-d330-5570-b420-cb94592b87ee","task_key":"default--validation--21329","task_revision_id":"1","upstream_id":"21329","short_description":"HellaSwag validation 21329","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to keep mosquitoes away [title] Wear long sleeves, pants, and closed shoes. [step] Mosquitoes are attracted to sweat and the natural bacteria that congregate on your skin. Keeping yourself covered in long sleeves, pants, and shoes will reduce mosquito attraction and also make it harder for them to bite you.\",\"endings\":\"['Wearing brightly colored shirts, shorts, and hoodies in your bedroom or classroom are also good options. [title] Consider protecting your skin from the elements.', '[substeps] Always wear long pants and closed shoes when traveling. Wear fabric made for mosquitoes, like other fabrics such as nylon or a natural fabric like wool.', '[substeps] Many outdoor superstores carry lightweight outerwear, so that wearing long layers is still cool and comfortable even in extremely warm temperatures. Choose light-colored clothes, like whites, tans, and pastels.', '[substeps] Stay away from casual, well-lit clothing when you can. Wear dresses instead of long-sleeved shirts.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}