{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7b489bc1-be63-59a6-91cd-74a6938654c4","task_key":"default--validation--21875","task_revision_id":"1","upstream_id":"21875","short_description":"HellaSwag validation 21875","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to create your own wiki with wiki site [title] Think of what your wiki's topic will be. [step] It can be a topic you're interested in, such as gaming and agriculture. [title] Choose what type of wiki you want to create.\",\"endings\":\"[\\\"[step] To make it easy to find, find wiki listings like your immediate advertiser's name, your number of threads, about wiki users, your blog, about wiki owner's status. [substeps] You will have to be grateful enough.\\\", '[step] The layout may seem strange, but when building your wiki, you will want to use a choice of wiki type from the above tips. [title] Choose the wiki you desire, as well as its requirements.', '[step] There are two options : [substeps] Personal wiki-you can create this type of wiki for free, and choose which pages are open for anyone to edit and which ones are for viewing only. Advanced wiki-there are no advertisements or control over who can access your wiki.', \\\"[step] Don't be afraid to build your wiki based on your pokerama's topic! No matter what you pick, if it is meant to be an fictional wiki it has to be made from real to be considered. [title] Most wikis will have an element for a wiki.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}