{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7346b5e4-254e-51fc-8827-af4de61c3a8a","task_key":"default--validation--12120","task_revision_id":"1","upstream_id":"12120","short_description":"HellaSwag validation 12120","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to decorate a dining table [title] Leave your table mostly bare. [step] Do not put plates or silverware on the table. Make sure there is enough space on the table for you to use for cooking, writing, reading, and other daily activities.\",\"endings\":\"['[substeps] By allowing your dining table to be mostly free of decorations and clutter, you can use the space for more than just eating! You can use wax or another product with a glossy finish to keep your table gleaming and looking great. [title] Lay a tablecloth or runner in the center of the table.', 'You should also refrain from putting unnecessary hooks in the decor, such as hooks for pans, plates, and utensils. By letting your table decorate, you reduce the risk of hooks getting smudged or worn.', 'For example, instead of \\\" it\\\\'s great when rain waves break tomorrow, \\\" you can write \\\" rain lowers the temperature around our house on the weekends. \\\" in addition, it\\\\'s okay to avoid putting your silverware/plates, pots, and pans on your table.', 'Place foods left at the table on the floor, to sit on, or to place non-dominant guests on opposite corners of the room. This will make it more inviting for them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}