{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2397238d-2928-5258-adaf-2a2b1ee591db","task_key":"default--validation--37977","task_revision_id":"1","upstream_id":"37977","short_description":"HellaSwag validation 37977","config":"default","split":"validation","body":"{\"activity_label\":\"Philosophy and Religion\",\"ctx\":\"[header] How to conduct a séance [title] Gather the participants. [step] Three is the minimum number of people who should participate in a séance. [title] Choose a medium.\",\"endings\":\"['[step] Choose between 60 and 100 people, depending on your chosen area. Léance typically spans around 30-55 minutes, depending on the location.', '[step] Usually, people commit a majority of their effort to the séance, but a few select the best course for the entire experiment, as well as for the potential participants. The medium should be strong, very persuasive, and large enough to move the selection tables.', '[step] This will be the person who will lead the séance and should be someone who has experience or psychic abilities. [title] Decide on who you will be summoning.', '[step] There are different types of mediums available but there are the following : [substeps] Rouge extract. Rouge extract is for people seeking to cure serious conditions except for autoimmune disorders.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}