{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"61c5e007-710d-57ba-843b-93c3c74c815c","task_key":"default--validation--208","task_revision_id":"1","upstream_id":"208","short_description":"HellaSwag validation 208","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to find excellent day care [title] Begin as soon as possible. [step] Many child care centers have long waiting lists, so it is never too early to start looking. If you are pregnant, start looking for child care for after your baby is born.\",\"endings\":\"['If you are married, start looking right away. Find an attendant or friend at your local hospital to help you get the paperwork started early.', '[title] Determine whether you are eligible for child care subsidies. [step] Often there is a waiting period for these funds, and they are only accepted by certain child care providers.', \\\"You should also look into outpatient child care after labor, as you may save a lot of money to simplify your baby's routine as well as make care more affordable. [substeps] If you decide to try outpatient, you should consider pursuing a lifestyle with a plan of life to meet your baby's needs.\\\", 'While this may be ideal, you need to start looking around only when you are currently in labor. If you are planning to go maternity, start looking right now.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}