{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"906fd5e6-0a08-5803-aaf5-5af3bf9a6b65","task_key":"default--validation--14682","task_revision_id":"1","upstream_id":"14682","short_description":"HellaSwag validation 14682","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to annoy people around you [title] Lick your food during a meal. [step] This is a great way to gross out and annoy your friends. Make a show of loudly and sloppily licking all of the food on your plate during lunch at school.\",\"endings\":\"['The food in front of you will probably look crusty and chewy. After you give in to this, turn it all in to the \\\" pass on \\\" trick.', 'Pick with your fingers the food that hangs around your mouth, and lick off the excess. Another great way to annoy your friends is to lick a spoon that you put down and breathe from it.', \\\"Joke around with your friends that you don't want any of them to steal your food, and lick it so none of them want to touch it. [substeps] If you're having dinner at a friend's house, you probably shouldn't lick your food there.\\\", 'This is a great show of respect for your family, especially if they seem really offended. [substeps] If you mess up while eating, it may be up to a teacher, principal, or parent to help you clean up without having to put your fork down.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}