{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cf1ffba9-601b-538a-92b8-c7dfaf044bdc","task_key":"default--validation--37009","task_revision_id":"1","upstream_id":"37009","short_description":"HellaSwag validation 37009","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to lead like thomas jefferson [title] Broaden your knowledge. [step] A polymath by nature and experience, jefferson spoke five languages, and was passionately interested in philosophy, science, religion, architecture and invention. He was well read and always had a thirst for knowledge.\",\"endings\":\"['He took care of nearly every aspect of life, when related to the world. Jefferson came from an era of poor powerlessness, and saw more harm, suffering and misfortune than he could see-which makes him unique and meaningful.', \\\"Learner in particular helped sketch a spoken article for a class, and therefore taught a broad topic. [substeps] This doesn't take long though, and if you know a very bright vocabulary of letters, your knowledge will be as well.\\\", 'This encouraged the student to pursue a dream to be a plumber, and got a reputation for being a skilled and carefully formulated designer. For him, the intelligence was great; his relationship with family, the flow of daily life and family dynamics were beautiful.', 'Devote at least one part of your week to reading books, articles, journals, newspapers and other reputable sources of information in order to keep up with developments in your own field and beyond. [substeps] Never assume that you are \\\" above \\\" digesting popular information sources as well.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}