{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"102e9662-2622-5c97-a841-1e012c23e7f6","task_key":"default--validation--32058","task_revision_id":"1","upstream_id":"32058","short_description":"HellaSwag validation 32058","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear hunter boots in winter [title] Go with a knee-high hunter boot for maximum protection. [step] Knee-high boots not only keep your feet dry but they also keep the bottoms of your pants from getting wet. These are the best options for winter footwear if you want to wear hunter boots.\",\"endings\":\"['[substeps] While you can also purchase ankle-length hunter boots, these will not protect your feet from the rain or snow as well as knee-high boots. [title] Choose dark colored boots in the winter.', 'The most common type of hunter boot is the trench-shaped boot, which has a lightweight sole and no lacing. This makes it a great choice for winter footwear.', 'A pant leg is a good option if you are only going to step on your feet a bit, as they are less likely to get wet, and your ankle boots will cushion your knees from the cold. [substeps] Wear hunter boots for most sports, including ice hockey and snow hockey.', 'They are also especially good for usability during fast or rocky terrain. [title] Look for a sturdy boot if you plan to skate.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}