{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51b06a68-4ad9-5746-9c99-d8b6cb5c2bfc","task_key":"default--validation--14764","task_revision_id":"1","upstream_id":"14764","short_description":"HellaSwag validation 14764","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to be confident if you have acne [title] Focus on your beauty. [step] Everyone has minor imperfections, but they also all have good features. Instead of focusing on your acne or scars, think about what makes you beautiful.\",\"endings\":\"['Think about your skin tone and whether your complexion makes you shine. [substeps] Try to always correct any negative thought or comment that pops into your head as you are trying to self-assess.', 'Write a list of what you find beautiful about yourself. Some ideas include : [substeps] \\\" i have pretty eyes.', \\\"It doesn't matter what school you go to, you don't have to be super toned or obsessed with your skin. Get a haircut that you're great at or a good hairstyle you love.\\\", \\\"If you can't focus on anything but your beauty, you're not who you are. Beauty is about correcting yourself where you need to.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}