{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4787529b-f30a-5515-a0fe-9359852f43a3","task_key":"default--validation--7405","task_revision_id":"1","upstream_id":"7405","short_description":"HellaSwag validation 7405","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to stretch plastic shoes [title] Wear thick socks and put on the shoe. [step] Wearing thick socks before putting on the shoe will give more room for it to stretch. Wear one or two pairs to fit into the shoe snugly.\",\"endings\":\"['[substeps] If you plan to wear the shoes for hours, coat them in a thin layer of plastic before you put on the shoe. This will help keep them from cracking when you put the shoe on.', \\\"[title] Know that the easiest way to stretch shoes properly is to just put them on briefly, toe first, then slide your foot out. [step] Don't wear your shoes with blood stains on them.\\\", \\\"[substeps] It's best to go for a quality pair of socks with a thicker sole so you can stretch the shoe without straining the shoe. Choose opaque or soft socks as your socks to improve the stretch.\\\", 'If the shoes have zippers, zip them up as far as they can go. [substeps] Wear thick wool socks for the maximum amount of stretching.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}