{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4e94e822-c013-545d-8173-c02fc860982f","task_key":"default--validation--9755","task_revision_id":"1","upstream_id":"9755","short_description":"HellaSwag validation 9755","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to buy an exercise bike [title] Estimate how often you will use the bike. [step] Upright bikes are easy to use and provide great cardiovascular workouts. They work abdominal muscles since you are keeping the body upright, and they offer upper body arm workout.\",\"endings\":\"['[title] Choose an upright bike for low impact cardio. [step] A spin bike most closely resembles the \\\" real \\\" activity of riding a road bike.', 'They get you working less because you are doing cardio. [substeps] Running is another great way to warm up your lungs and run faster.', \\\"Your bike may not provide the same muscle tone as stationary bikes, but you will run 1-2 miles (3.0 km) of cardio a day. [title] Read reddit's online workout chapters to learn more.\\\", 'You will want to check with your trainer before you begin to ride the bike. You might also want to think about shipping your bike.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}