{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1bdfd800-32a8-5cc5-94e3-615366fe5432","task_key":"default--validation--27927","task_revision_id":"1","upstream_id":"27927","short_description":"HellaSwag validation 27927","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to choose a bridle for your horse [title] Decide on the type of material you want. [step] Bridles vary in price range based on the quality of the material. The cheapest bridles tend to be made of synthetic leather, while better bridles will be made from a higher quality sedgewick or uk leather.\",\"endings\":\"[\\\"[substeps] Wood bridles are highly durable and tend to last longer than smooth wood bridles, so they're best for scenery ride rides. Green bridles are more durable, but often advise caution against riding on red bridles.\\\", 'The density of the material makes the crown top a bit harder to pick. [title] Choose a bridles unique to your specific horse.', '[substeps] Consider the function of your horse when deciding how much to spend on materials. You may want to splurge and get the best leather for your top show horse.', \\\"The quality of the material will also affect how much you can afford to spend. For instance, for a hard to manipulate saddle, you may want to buy a soft, lightweight or leather bridle that doesn't touch any area of the horse.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}