{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"05ca0520-2343-5a91-b88e-0e01fd7ea21e","task_key":"default--validation--47386","task_revision_id":"1","upstream_id":"47386","short_description":"HellaSwag validation 47386","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to fit in at high school [title] Start early. [step] Most high schools have an orientation period where you can visit the school. While you're at orientation, try talking to other people to see if you have any common interests.\",\"endings\":\"['[substeps] For example, you could say, \\\" hi! I\\\\'m john. So are you starting here, too? I\\\\'m going to be in the band.', \\\"[substeps] Listen to your teacher and try to make small talk with other students. Make your first day of sixth grade a friendly experience, especially if you're not the outgoing type.\\\", \\\"Try to get a degree in technology or math and find a major in sports or music to beat your high school's dress code. You want to make sure that you make appropriate friends who like you for who you are.\\\", 'You could compare classes by saying something like, \\\" i really like physics! Would you mind meeting up this year in the fall? \\\" or \\\" i\\\\'m new at english. Are i at that level? \\\" [substeps] As you get older, it\\\\'s important to make an effort to update your reading level.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}