{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"69b39040-6227-54e1-b789-5df3cc38aad3","task_key":"default--validation--27212--variant-9d4c2f69cce4","task_revision_id":"1","upstream_id":"27212--variant-9d4c2f69cce4","short_description":"HellaSwag validation 27212","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to make everyone smile [title] Make eye contact. [step] Smile and act friendly. When someone is talking to you, make eye contact to show that you are interested.\",\"endings\":\"['While faking a smile may seem like a simple \\\" hello \\\" expression, it actually looks cute and will make people smile. [substeps] If the person who is talking to you begins to talk, smile.', \\\"Don't over-extend your gaze or look away from the person who is talking. This will make them feel like they have to cross you off the conversation.\\\", 'This means both of you are engaged with what the other person is saying, and that you are paying attention to them. [substeps] Strong eye contact says that you are listening to the other person, not the other way around.', 'Eye contact is a way of connecting with another human being on an intimate level, and is key to being well-liked by others. [substeps] This means putting away your phone and being present in the moment.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}