{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d4217439-5d31-5131-bea1-29827f8459b6","task_key":"default--validation--35479","task_revision_id":"1","upstream_id":"35479","short_description":"HellaSwag validation 35479","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to care for a facial piercing [title] First, decide what piercing you want. [step] There are many to choose from. It's up to you to decide which is best.\",\"endings\":\"[\\\"It is best to ask your piercer what kind you want. [title] Consider buying one that's close to your size.\\\", \\\"Some piercings will be an oval and some will be round. Over time, you'll then need to decide which ones are best.\\\", \\\"[title] After you get your piercing, make sure to listen to your piercer's advice. [step] Contrary to popular belief, these people know exactly how to take care of a piercing.\\\", \\\"Don't get too stuck on what kind of piercing you want. Just apply products to make sure you cover all the surrounding areas.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}