{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3bc66a91-60df-5df6-944f-f323fb663ee7","task_key":"default--validation--5015","task_revision_id":"1","upstream_id":"5015","short_description":"HellaSwag validation 5015","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to start an antique pocket watch collection [title] Attend auctions and estate sales that are selling pocket watches. [step] There are websites online that will give you the location and times for live auctions in your area. Many modern day auctions will also give you a list of items that are being auctioned that day with photos and details.\",\"endings\":\"['[substeps] In some cases, you may be able to sell your pocket watch on the first day, but in others you may not feel comfortable selling your watch on the first day. To avoid this, travel to all of the major estate sales that you have in the area.', \\\"Attending a auction will allow you to see the expected numbers of pieces before purchasing one. [substeps] Some auction websites aren't looking for cheap watches, often requiring individuals to bid for a variety of silver period watches.\\\", 'Make sure to search for the keyword \\\" pocket watches \\\" to find events that you can attend. Your goal early on is not to bid or purchase any watches, but see what buyers are currently looking for and how much they are willing to spend.', 'Buying pocket watches is a great way to get to know the market first. Just be aware you may not be able to make the auction and estate auction rounds regular.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}