{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2f408406-fabb-5411-a1cc-bc6a9064ecb1","task_key":"default--validation--30263","task_revision_id":"1","upstream_id":"30263","short_description":"HellaSwag validation 30263","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to diagnose testicular cancer [title] Check your testicles once every few months. [step] Regular self-exams can help detect abnormalities due to cancer or other medical issues. Blood vessels, tubes that carry sperm, and other normal parts of your anatomy can feel strange at first.\",\"endings\":\"[\\\"In time, you'll learn what's normal for you, and you'll be able to recognize anything unusual. [substeps] If you want help understanding your normal anatomy, ask your doctor for guidance at your next check-up.\\\", 'Be sure to have these also checked every month and right before or while you are exercising to detect symptoms of testicular cancer. [substeps] If your doctor examines your testicles, they will know if they look abnormal.', 'Your body\\\\'s \\\" pressures \\\" can leave a hole in the fabric of your testes, causing pain. If you\\\\'re concerned about having these openings, consult your doctor or your gynecologist.', 'Pay attention to these changes so you know when to expect them and when to see it. [title] See if you or a partner feel pain or tenderness while urinating or using the restroom.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}