{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a701da41-145c-55eb-a2cc-c4688c2e1ae6","task_key":"default--validation--12423","task_revision_id":"1","upstream_id":"12423","short_description":"HellaSwag validation 12423","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to trace your ancestry [title] Ask questions. [step] Find out as much as you can about your family history. Look through old photo albums, and ask your family members what they know.\",\"endings\":\"['Also, keep in mind who you are. Do they know where you come from? Who do they know in your country? Are there any family members you think might be purebred? An ancestor may have shared heritage with a much larger family.', 'Ask for the names of grandparents, great-grandparents, and other relatives, as far back as they can remember. Determine the names, locations, and timelines of the ancestors you want to know about.', 'It can take time, but it is usually worth it. Think about how it would feel to have a history with any family member that you find interesting.', \\\"There will be loads of information on every ancestry you can trace. This may not be useful in your research if you are going to take out someone else's ancestry, but if you have limited information, you can use a more specific method.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}