{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c31ef2bd-6e79-5127-8942-d5e302377e43","task_key":"default--validation--19011","task_revision_id":"1","upstream_id":"19011","short_description":"HellaSwag validation 19011","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be a christian when your spouse is an atheist [title] Know that atheists are people just like christians. [step] Some are nice, some are not. Some are happy and successful, some are not.\",\"endings\":\"[\\\"It's up to you as to what to do. [title] It's time to find your religion.\\\", 'Because they believe in god and will do whatever they want to do. [title] Understand that god is only one thing.', 'Never assume anything about a person because of their religion, spirituality, or lack thereof. [title] Continue to attend church regularly on your own.', \\\"Some are miserable, some are relaxed and lazy, some are nice, some are harsh and just plain awful, so whether you join the holy qur'an or think that must be completely optional, they are christians just like you. [title] Remember that everyone has his or her way.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}