{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5f9c2a4-e421-5d60-80ba-928abea312a7","task_key":"default--validation--1713","task_revision_id":"1","upstream_id":"1713","short_description":"HellaSwag validation 1713","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to hang voile [title] Use the voile by itself or with a second curtain. [step] Many people use voile curtains in conjunction with heavier standard curtains to create a visually appealing layered look. Others use voile curtains on their own.\",\"endings\":\"['Determine which option works best for your room before you do anything else. [substeps] Voile is lightweight and transparent, so it will never provide the full privacy a normal set of curtains would offer.', 'This creativity can be transformed into an art can-do. [substeps] For instance, you might decorate two or three corners of your bedroom using the voile by yourself.', 'Decide what color scheme is best for you. Opt for using either matte or glossy voile, opting for an inexpensive color if you want to create a very dressed-up look.', \\\"[title] Choose a color scheme that's pleasing to you. [step] If the voile measures the overall color scheme of the home, then you'll need to draw from that color scheme.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}