{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8dce4f54-ce11-5cf1-bda5-7dc5a94b1c43","task_key":"default--validation--8203","task_revision_id":"1","upstream_id":"8203","short_description":"HellaSwag validation 8203","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be awesome [title] Develop a talent. [step] Whatever your talent is, pursue it. People who become \\\" awesome \\\" at what they do invest a lot of time into getting there.\",\"endings\":\"[\\\"[substeps] You don't have to make it something you're not good at for it to be cool. Getting it doesn't have to be a big thing.\\\", \\\"Who ever said that being awesome was easy? [substeps] Some theories say that it takes about 10 , 000 hours of practice before you can truly master something. Now, that's a lot of hours, and it's totally possible to have an awesome talent before even 1 , 000 hours of practice.\\\", \\\"Being fantastic is validated by being in the least amount of trouble, but have some backbone by going beyond the classroom teacher. If you do have trouble in a class you're struggling with, sit next to someone who cares about it and give them a boost.\\\", \\\"Therefore, you have to show up for classes first, get excellent grades, volunteer at foundations, go to parties, etc. [substeps] Make a high school effort, even if you don't know the subject.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}