{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1c9a22b5-62fc-57c6-88aa-f347745db685","task_key":"default--validation--35661","task_revision_id":"1","upstream_id":"35661","short_description":"HellaSwag validation 35661","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to raise a gentleman [title] Promote good listening skills. [step] A gentleman knows how to listen to others. Teach him to wait his turn to speak instead of allowing him to interrupt others during a conversation.\",\"endings\":\"['If another participant is struggling, talk them out of it. [title] Show respect for others by making eye contact, turning away when appropriate, and being persistent.', '[title] Give your guest time to speak and cut him off with breaks as they try to interject. [step] Hold your lip on the very edge of your mouth, but maintain eye contact and nod.', 'For instance, give him the silent treatment when you need to maintain a firm but kind tone in a conversation. You can also give him the verbal treatment during dinner, and keep it minimal.', 'You also need to give your son the courtesy of listening to him when he is trying to tell you something. [title] Instruct him on being friendly and personable.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}