{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87b23c21-a4de-5b72-8988-4f5677834a05","task_key":"default--validation--12268","task_revision_id":"1","upstream_id":"12268","short_description":"HellaSwag validation 12268","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to be the only single one in the group [title] Go easy on the conversation. [step] The worst feeling as a third wheel is never knowing how or when to be a part of a conversation between two romantic partners. Sometimes it'll feel like you're stuck on somebody's else date, and maybe you are.\",\"endings\":\"[\\\"So don't be afraid not to participate when it doesn't feel right. [substeps] Naturally, always speak when spoken to.\\\", \\\"Forcing yourself to want people who already have your best interests at heart and then suddenly leave your ball in someone else's lap is a major deal. If you need to find something to engage in, focus on conversations with your mates and joking around.\\\", 'Consider not only yourself, but the other people in the group, too. [substeps] Consider having a little chat every now and then with a special someone.', \\\"Whatever you do, be as upfront as possible about what you're thinking about, feeling, and what you can do about it. [substeps] If you find yourself thinking that you're the only single one in the group, you're not yet.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}