{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9752386f-2817-5978-9794-4bcd55d2f06c","task_key":"default--validation--18342","task_revision_id":"1","upstream_id":"18342","short_description":"HellaSwag validation 18342","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to wear a skort [title] Know the look. [step] The traditional skort looks like a skirt from the outside but has a pair of shorts tucked away on the inside. The pieces of material are connected at one or more seams, forming a full, single garment.\",\"endings\":\"['Most skort pieces are lightweight with a jacket for a weather-appropriate look. [substeps] Dress for colder weather: standard skort pieces are made from thin cotton, nylon, and merino wool.', '[substeps] Skort trousers work well for this look. A high-waist dresses is a fashionable alternative.', '[substeps] Most traditional skorts are standard short-length or miniskirt-length, so the lower hem falls out anywhere from the mid-thigh to the top of the knee. You may find a few that fall out at or just below the knee, but you will rarely if ever find one that reaches maxi skirt lengths.', 'This means that you should spend much of your day wearing a skort that has two sides similar to each other. This hue tends to coat most of the outfit and unevenly dressy and beautiful.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}