{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"774c59fe-306b-5dd6-b5b0-bdb7ac1f0971","task_key":"default--validation--16874","task_revision_id":"1","upstream_id":"16874","short_description":"HellaSwag validation 16874","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to interrogate someone [title] Act friendly and casual. [step] Studies and empirical evidence have consistently shown that the best way to get a confession from a subject is to make them feel comfortable with you. They need to trust you.\",\"endings\":\"['[substeps] From an early age (mid-teens), talk with your subject about subjects like their school, their work work, and their personal lives. See how your subjects respond to the person by asking them questions.', \\\"[substeps] If you're trying to gain that extra friend feeling you've created, you have to get that friend over in a friendly manner. After talking, as you present more flirtatious body language, you'll sound brighter and more confident.\\\", \\\"It isn't appropriate to scold or tease people, even if the subject is someone you don't know well. Don't make an effort to prod them or pull them away.\\\", \\\"You will not get this if you act like the evil nazi in some hollywood movie or a bruce willis style cop. Act like someone who's generally easy going and just trying to do their job, and you'll be much more sympathetic to the person you're interrogating: the first step in getting them to trust you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}