{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c97469e3-b323-531c-82dc-dd0cee6f444d","task_key":"default--validation--26810","task_revision_id":"1","upstream_id":"26810","short_description":"HellaSwag validation 26810","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to send a flirty text message [title] Text like you mean it. [step] Your genuine interest in another person is the most attractive and meaningful message you could possibly send. Don't play games by being intentionally vague.\",\"endings\":\"['If you send a flirty text and then leave it to someone else to get it, they will get the message because then you\\\\'ll get nothing in return. Also, you don\\\\'t have to omit something when you are texting, like \\\" hi guys, thanks for looking at me.', 'Text one of the least interesting messages a few days afterward to your friend, or even one first thing in the morning just to make sure she knows you like her. [substeps] For instance, if you see your friend checking her teeth, point out that she needs braces just like you.', \\\"Read all of your texts to yourself before sending them. Ask yourself if it'd be a text you'd be happy to receive.\\\", \\\"Your text message will convey to your recipient how you feel. [substeps] Of course, this doesn't mean not being open and honest.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}