{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"614ef7dc-9fbf-53c4-955e-5faca17edc50","task_key":"default--validation--20026","task_revision_id":"1","upstream_id":"20026","short_description":"HellaSwag validation 20026","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be certain that your choice to not marry is sound [title] Read articles online about marriage experiences, from both sides. [step] Try not to read about celebrity couples, as those marriages are hardly realistic. Find blogs, vlogs, or advice articles.\",\"endings\":\"['[title] Research events before making your choice to not marry. [step] By researching everything you can think of and interviewing sources that can help you form your own beliefs, ideas, and opinions, you can determine if your decision is realistic.', 'If something seems too off, you\\\\'ll have plenty of fodder to defend against it. [title] Recognize that there is more to your decision than just a general opinion (\\\" that\\\\'s one thing \\\").', \\\"Sign up for these sources on an iphone and watch videos of celebrity couples that you can indulge in, or read about different spouses from various magazines. It's good practice to stay informed, especially if you're not watching tv, while reading.\\\", \\\"Find things written by people who don't want to get married, and why they chose not to get hitched. [title] Talk to both married and unmarried friends and relatives, and ask them why or why not they chose to marry.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}