{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5ff6a5d9-cf97-512b-ba1b-4fe64c18b639","task_key":"default--validation--39785","task_revision_id":"1","upstream_id":"39785","short_description":"HellaSwag validation 39785","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to choose a paintball gun [title] Research what is available. [step] If you're new to the sport, you may not have any idea how many options you have. Look at some sporting goods store sites to get an initial idea, and look at paintball specific retailers.\",\"endings\":\"['Read reviews on various models to gauge which guns people tend to like most. [substeps] With so many choices on the market, it\\\\'s really not worth it to try and find the \\\" best \\\" gun, rather to find one that you think will suit you best.', \\\"It's a good idea to bring a smartphone or tablet, along with a flashlight and a paintball shield, just in case. You can also research your options online, just in case.\\\", \\\"These are usually more knowledgeable than even the people who own paintball guns, so you're less likely to be disappointed with the way different cameras work. [substeps] These are usually easier to buy than magazines, but also expensive.\\\", \\\"Many vendors offer their guns online, especially if you are selling paintball toys or playing with the tiller. If you decide you don't like the product, ask a store worker to quiz you about what the shelf life of guns is and who stocked that shelf.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}