{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"41380d47-1ba4-5660-a0ab-12baa81da7f2","task_key":"default--validation--35695","task_revision_id":"1","upstream_id":"35695","short_description":"HellaSwag validation 35695","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to turn jeans into shorts [title] Choose a pair of jeans to turn into shorts. [step] The best jeans to choose will fit you comfortably in the hips, butt and thighs. Remember that baggy jeans will become baggy shorts, and tight jeans will be tight shorts.\",\"endings\":\"[\\\"[substeps] When choosing jeans, choose pair that are relaxed and nice-looking, rather than tight. Skinny jeans work well because they're so baggy and let in some of the heat.\\\", 'Look at the weather forecast when deciding if jeans should be combined into your plan. [substeps] Below are just a few good brands of jeans.', 'Focus on wearing the right type of jeans to avoid huge setback. A snug fit makes them difficult to stretch on the knees, butt, and thighs.', \\\"[substeps] Jeans made with stretchy material are not the best candidates to turn into shorts. They usually have pieces of rubber or plastic woven in with the fabric, and those won't look good dangling from the bottom of the shorts.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}