{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"18343d72-e35d-5ca9-8ff6-443bd087d593","task_key":"default--validation--21081","task_revision_id":"1","upstream_id":"21081","short_description":"HellaSwag validation 21081","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to become an internist [title] Take science and math classes in high school. [step] Enroll in challenging advanced placement (ap) classes whenever possible and prioritize subjects like biology and calculus. Improve your communication skills, which you'll need to interact with patients, by taking composition and speech courses.\",\"endings\":\"['Take these skills into the classroom and without a doubt, invite potential patients to join you in classes. [title] Volunteer at a local nursing home or hospice facility.', \\\"Additionally, enrolling in a science or math tutor service will give you access to various notes if you see a patient struggling in their class that you trust. [substeps] When you don't pass it on, just pay attention to the person's responses.\\\", '[substeps] Get the highest grades that you can, as your college scholarships will be partially based upon your gpa. [title] Volunteer at a local hospital or health clinic.', '[substeps] Take advanced placement courses to expand your knowledge. [title] Get a doctor or nurse referral.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}