{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ab5d6a80-0e03-5dfb-8447-ca9f619c2c4b","task_key":"default--validation--43845--variant-fce407b2fe02","task_revision_id":"1","upstream_id":"43845--variant-fce407b2fe02","short_description":"HellaSwag validation 43845","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to act around your grandpa [title] Have respect. [step] It is crucial to show respect to all people, but particularly to anyone in your family, and most particularly to family members who are older than yourself. Your grandfather is very wise compared to you, and has had a great deal of life experience.\",\"endings\":\"['Do not expect your grandfather to follow you around like a longlost god. Learning how to respect yourself does not mean being rude or disrespected; doing so shows that you care about yourself and wants to help others.', \\\"Demonstrating love to and giving respect in return can make him a useful grandparent and a great grandfather. [title] Make peace with your grandpa's decisions.\\\", 'This means he probably knows a lot more than you do about a great many things. Therefore you should listen to what he has to say, and take it very seriously.', \\\"You want to be kind and considerate of anyone you meet so you can be a good leader in society. [substeps] Respect your grandfather's age, but also respect that your mother might be older than him and this is so you can be an effective leader in your grandchildren.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}