{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25dac4e4-ea82-5741-875d-5d85f091b633","task_key":"default--validation--18361","task_revision_id":"1","upstream_id":"18361","short_description":"HellaSwag validation 18361","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to prevent lead poisoning [title] Determine when your home was built. [step] Extra precautions must be taken to avoid lead poisoning if your home was built before 1978. Risks of lead in older homes include paint and pipes.\",\"endings\":\"['[substeps] Libraries may also be dangerous if you live in a metropolitan area with a moderate population of people. You should also keep older buildings out of reach and away from children and large groups who are allergic to certain foods.', \\\"[substeps] Find out when your home was built by looking at the ownership records. Your home's title policy, title search, and purchase appraisal should all have the year the house was built on it.\\\", 'A large number of people are at increased risk of lead poisoning due to lifestyle changes such as traveling to work and subjecting clients to a high-pressure environment. [substeps] Conduct hourly surveys, receive federal and state reports, and purchase safety equipment.', '[substeps] Wear your safety glasses whenever you are inside. You must also wear gloves when handling lead.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}