{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e812727c-3acb-5328-a8a4-40522702fe67","task_key":"default--validation--22270--variant-e3ec361bdd06","task_revision_id":"1","upstream_id":"22270--variant-e3ec361bdd06","short_description":"HellaSwag validation 22270","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to describe to a hair stylist what hair you want [title] Determine what type of hair you have. [step] Is it long or short? Curly or straight? Thick, baby-fine or somewhere in-between? [title] Make the appointment. [step] Let the receptionist know that you'll need extra time to discuss how you want your hair.\",\"endings\":\"[\\\"Tell them when you'd like your hair and ask for a time estimate. Be sure to hang up your resume so that you don't lose track of time.\\\", 'Tell the receptionist if you have short hair of any type at the time of payment. [title] Supply a copy of the \\\" color wheel \\\" you provided.', 'This is true even for walk-in salons. [title] Bring a picture! [step] Find a hairstyle and/or color you like in a magazine or on the internet.', 'You need the appointment to be 30 minutes earlier than expected or there may be complications later. [title] Check up with your doctor or stylist first.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}