{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5cbe031d-1d61-5098-a3b9-dde8391dbd48","task_key":"default--validation--10559--variant-fd6c37f8d144","task_revision_id":"1","upstream_id":"10559--variant-fd6c37f8d144","short_description":"HellaSwag validation 10559","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to make a kilt [title] Choose a tartan by clan. [step] Clans and large families of scottish origin often have their own tartan patterns since the early 1800s. You can wear a clan pattern only if your family has current or ancestral ties to that clan.\",\"endings\":\"['Choose a tartan pattern to conceal your personal personal symbol. [substeps] In scotland, manian and claymores are the colors commonly associated with tartan patterns.', '[substeps] Find out which clan you belong to. As long as you know your surname or a surname connected to scottish ancestors, you can search online for your clan name.', 'Although tartan patterns are usually found in highland territory, there are still at least three popular patterns : [substeps] The iu family tartan and country tartan. This may seem strange, but ask yourself if you enjoy the highland style and otherwise avoid territory and local areas.', \\\"If you're interested in getting a tartan from another clan, get a pattern online or speak to a village guide with a tartan for more ideas. [substeps] The scottish clan often has patterns as well and is a very ancient, modern clan, extending generations over the centuries.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}