{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9b150a1a-eea6-5d84-986c-0ac993c44c9b","task_key":"default--validation--27866","task_revision_id":"1","upstream_id":"27866","short_description":"HellaSwag validation 27866","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to make money selling at festivals [title] Scout the show ahead of time. [step] Before vending a show, take time to scout it out. Go and see what people are selling.\",\"endings\":\"['Figure out how heavy the merchandise is and what the brochure clearly states and how many people are present at the show. You can also focus more on making money on the show itself as opposed to the band.', \\\"If you enjoy reading and listening to music, go ahead and get tickets to those shows. If you're trying to be successful at selling at festivals, you'll need to find a place to hang out at.\\\", 'Check out the quality of work, evaluating whether your products are of similar quality and whether or not they fit the general feel of the show. For instance, edgy, experimental art is not likely to fit in at a county craft festival.', 'Find out what the stores are selling, what the artists are doing at the show, and whether/when you can start making flyers or postcards. [substeps] Follow the conventions of festivals in your area and use this knowledge to make a flyer or postcards that will attract smaller crowds.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}