{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3fc26106-da75-5839-98bd-4bd4375722a6","task_key":"default--validation--816","task_revision_id":"1","upstream_id":"816","short_description":"HellaSwag validation 816","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to organize your jewelry box [title] Empty out your jewelry box. [step] The best way to start organizing your jewelry box is to empty it completely. This will allow you to see everything you have and sort it out, so when you put it back into the box it will be organized.\",\"endings\":\"['[substeps] If you have a large or unusually sized collection, place it in a different room. Storing it this way can prevent you from seeing everything you have or messing with it later.', '[substeps] Use a vacuum to rid your jewelry box of dust or debris so that you can see everything in the box. You can also divide your jewelry box into rings and necklaces if necessary to make it easier to see what you have.', \\\"[substeps] Don't just empty out your jewelry box in a garbage can. Also don't haphazardly shove your jewelry box with its contents right in front of your bed.\\\", \\\"[substeps] You may want to lay down a towel on your dresser or table before you empty out your jewelry box so that neither your jewelry nor dresser gets scratched. [title] Untangle your jewelry if it's knotted together.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}