{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ca52f510-9f98-5e2b-bfa2-37c7ae7a3daf","task_key":"default--validation--9618","task_revision_id":"1","upstream_id":"9618","short_description":"HellaSwag validation 9618","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to perform a card force trick [title] Find your audience. [step] You'll need someone to perform your magic trick for. Search out a group of people to amaze.\",\"endings\":\"[\\\"A magician's trick is usually only performed by about 1 person if the audience is large enough. [substeps] For best results, have a group of people do it for you.\\\", 'This can be teachers, older students, people in familiar or historic settings, such as a place of worship, etc. [substeps] If you cannot find anyone, try to find the individuals in neutral attire and ask them to perform the trick for you.', 'If there are people not associated with the magic trick, identify them by eye. If there are too many, narrow down the list until you find a number at least 10.', '[substeps] Look for people who seem friendly and open to your performance. Avoid people who seem reclusive or groups in the middle of serious conversation.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}