{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"041007b4-ecc6-551e-9f11-af447a06ffdf","task_key":"default--validation--17462","task_revision_id":"1","upstream_id":"17462","short_description":"HellaSwag validation 17462","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to look great in sweatpants [title] Choose fitted sweatpants. [step] Sweatpants that have a tailored look are on-trend and turn the \\\" sweatpants are for slobs \\\" bias on its head. A slim fit and tapered leg are key for getting this look right.\",\"endings\":\"['[title] Choose leggings that cover only your leg. [step] Avoid baggy pants when choosing fitted sweats.', \\\"The pants will still have a casual, slouchy look, but they should fit more like a pair of trousers than the oversize sweats you wear when you're home sick. [substeps] Large, baggy, or oversize sweatpants generally aren't flattering.\\\", 'Sweatpants that sag at the ankle are downright trashy. [substeps] Wrap up your jeans so you can dress as you normally would.', \\\"[substeps] Sweats with vertical stripes that go all the way up to your knees are good sweatpants. If your legs are naturally curly or the pants are loose fitting around them, don't get them.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}