{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"80213bf5-7ff8-5875-9478-4cacddf0f85f","task_key":"default--validation--38602","task_revision_id":"1","upstream_id":"38602","short_description":"HellaSwag validation 38602","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get a smoke smell out of fabric handbags [title] Fill a sink or tub with cool water. [step] Warm water can shrink or damage the fabric of your handbag. Fill the sink or tub with enough water to fully submerge your bag.\",\"endings\":\"['Add some fabric softener and warm water to help soften the fabric, adding flavor and scent. [title] Submerge the bag for about 15 to 20 minutes.', \\\"If you have a bath, make sure it's not too hot to submerge your bag. Place your handbag inside of the water.\\\", 'Leave room to work the fabric with your hands. [title] Use a laundry soap which is created for delicate fabrics.', 'Leave the bag in the hot water for an hour or two. Add more lemon juice to make the water cool, but do not pour it in directly or the aroma may damage the bag.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}