{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"297b8663-c6ab-569a-9fc1-7e325242baa0","task_key":"default--validation--46293","task_revision_id":"1","upstream_id":"46293","short_description":"HellaSwag validation 46293","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to buy a seventh year wedding anniversary gift [title] Consider copper jewelry. [step] Copper is said to bring good luck, prosperity and good fortune to those who hold it, so why not let your spouse wear it? You can search websites like etsy and amazon to find vendors selling copper jewelry. You can find beautiful and unique copper pieces online, as it's a very popular material for jewelry.\",\"endings\":\"['Gold is also accepted, and a big color as is copper. Go thrift stores to find copper jewelry.', 'Make sure to check with your vendors before buying your jewelry, as copper jewelry is usually sold as a gift. [title] Find on-demand jewelry at jewelers.', '[substeps] Choose something that your partner would actually wear. If they love fun necklaces, add a copper one to their collection.', '[title] Buy hot water and take a shower. [step] High temperatures could lead to a tremendous fuss.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}