{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"134d001f-919c-5936-92ed-ffaccced1ba3","task_key":"default--validation--43737","task_revision_id":"1","upstream_id":"43737","short_description":"HellaSwag validation 43737","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to perform an intervention [title] Consult with a professional. [step] To ensure the best chances for success, you should consult with someone who has experience helping addicts and conducting interventions. A professional interventionist will be able to guide you and other family member through the process, increasing chances that the intervention will be successful.\",\"endings\":\"['Your obstetrician will also be able to help resolve any issues the doctor may have had in the past. [title] Conduct a call to a hotline.', '[substeps] You may want to contact a family member or partner who has had experience dealing with people in this ssd. [title] Keep them apprised of the situation.', \\\"You can meet with the professional beforehand to plan things out, and you may even want to invite him or her to the intervention to act as a facilitator. It's especially important to consult with a professional if the following is true : [substeps] The person has a history of mental illness.\\\", 'Ask your doctor or therapist for a referral. [substeps] If you need help locating a professional, use websites like navea or caresunnist.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}