{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c9ce9d3-e702-586e-bcc8-fcfb973ae705","task_key":"default--validation--16392","task_revision_id":"1","upstream_id":"16392","short_description":"HellaSwag validation 16392","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean a purse [title] Throw away trash once a week. [step] One of the most important steps to take towards keeping a clean, organized purse is frequently checking for and discarding half eaten snacks, gum wrappers, and expired coupons. It's easy to let scraps of trash build up in your purse, but try to throw them away more regularly.\",\"endings\":\"['Begin by putting away only : [substeps] Sugary candy candy sugary food stored in plastic baggies. Coupons and other high-priced items that are consumed continuously.', \\\"A good rule of thumb is to toss in trash when you also get other items, including small miscellaneous items. Don't forget the occasional little tip as well.\\\", \\\"[title] Vacuum out crumbs and dirt. [step] Even after you've gone through your purse and thrown out any trash, the inside lining of your purse may be dirty.\\\", 'The more often you open your purse, the less likely it is that you will run out of stuff to throw away. [substeps] Toss your trash into a trash can when you make a trip to the bathroom.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}