{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7a35a068-3a2f-51a0-bc49-a2833ebab1e5","task_key":"default--validation--24485","task_revision_id":"1","upstream_id":"24485","short_description":"HellaSwag validation 24485","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to become a gold dealer [title] Decide what kind of gold to buy. [step] You may decide to purchase scrap gold, jewelry or coins. [title] Research the competition.\",\"endings\":\"['[step] Scrap gold is marketed as being high quality, usually in equal quantities. This means there are so many different companies that are competing to get gold pricing.', '[step] Gold dealers are almost always selling their gold at auction. Auction sites often have simple policies about gold dealers being winners.', \\\"[step] Gold sellers know a lot about competition and just want to make sure they understand what others say and that they'll get the best dealer for the experience. Find reviews of the competition and talk with other gold sellers.\\\", \\\"[step] Check out the competition both in your geographic area and on online auction sites. Your will want to look for what the competition isn't offering so that you can provide your customers with unique value.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}