{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0e716646-184b-5147-99d0-dc9ebf4880a8","task_key":"default--validation--28338","task_revision_id":"1","upstream_id":"28338","short_description":"HellaSwag validation 28338","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to barrel race [title] Pick the right horse. [step] Your horse is the most important piece of your barrel racing team--other than you, of course. You and your horse should be equal partners and work together to learn to succeed at barrel racing.\",\"endings\":\"[\\\"You should : [substeps] Be free to throw flies over your horse's head. These well-timed throws can stun your horse, making him vulnerable to sharp bolts.\\\", \\\"In order to fail by running out of time, it is important to appear on the field on time, and listen to your horse as it races. You can use your horse in your attempt to permanently trigger it's own spotting.\\\", \\\"If you don't have confidence in your horse, stock up on kits and/or buttock leathers. [substeps] Additionally, winter-related races are great places to focus on the basics of barrel racing.\\\", \\\"Some horses are more capable of becoming barrel horses, so you'll want to choose your partner carefully. [substeps] The horse's health is the most important consideration.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}