{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f7d36990-7858-588c-ad94-12c2682a0bb9","task_key":"default--validation--14615","task_revision_id":"1","upstream_id":"14615","short_description":"HellaSwag validation 14615","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to become an equine appraiser [title] Get to know horses. [step] Professionals recommend dealing extensively with specific breeds of horses that you want to build a career around. Some equine appraisers focus on draft horses and other large breeds, while others narrow their interest to riding horses and other smaller, lithe breeds.\",\"endings\":\"[\\\"[title] Become savvy about legal valuations. [step] It's extremely beneficial for someone who wants to become an equine appraiser to read up on applicable law as much as possible.\\\", 'If you feel comfortable dealing with large breeds of horses, you can talk with adult horses at school or put them to work at horseback riding lessons. [substeps] Although riding is all about being able to ride, what you practice is tailored to the horse that you want to work with.', 'Because of these differences in temperament and general focus, it may be helpful to find partners that can accommodate that. [substeps] Do not ask first about working horses.', \\\"[substeps] Equine appraisers should already have the temperament to run an equine business. If you're interested in working with horses, trust your gut.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}