{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"deb1428d-b0db-5d7a-b3e0-6b45d2b5a3ab","task_key":"default--validation--21449","task_revision_id":"1","upstream_id":"21449","short_description":"HellaSwag validation 21449","config":"default","split":"validation","body":"{\"activity_label\":\"Holidays and Traditions\",\"ctx\":\"[header] How to collect cards at your wedding [title] Use a secure card box. [step] If you choose to collect cards using a card box, make sure it is secure. Choose a card box that has a lock.\",\"endings\":\"['Then, use your card box\\\\'s key combination to secure it. Put the key combination in the \\\" + \\\" on your card box or off to the side.', 'The card box should have a single slit in the top. It should remain locked throughout your wedding festivities.', 'This will make it easier to access your cards at the wedding. [substeps] There are a number of card boxes that you can purchase at your wedding shop, but any one that you choose needs to be secure.', 'This provides security for your guests who want to know the card contains secrets. [substeps] If you add valuables into a box, make sure there are locks at the bottom of it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}