{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7f626827-96cb-55b2-b885-8aed7725d0a0","task_key":"default--validation--48390","task_revision_id":"1","upstream_id":"48390","short_description":"HellaSwag validation 48390","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to choose a coffin [title] Visit a funeral home. [step] Funeral homes will almost always help you select a coffin from their stock. Most funeral homes will have a display room for you to select a coffin from.\",\"endings\":\"['If you are looking for a coffin to buy, keep an eye out for anything on display. Many funeral homes also have closets in the front and back.', \\\"You'll be able to see each coffin in person, allowing you to find one that is right for you. The funeral home staff can also help answer any questions you might have about a particular coffin.\\\", 'This room is usually around 17 or 30 feet (50 or 75 m) apart, so you can walk in with a clear head and comfortably sit in the room. [substeps] If you go to a funeral home, they will have a display room and have you select a coffin.', \\\"They'll fill this room to the brim with flowers. In addition, staff at a funeral house will often also assist you if you have certain needs.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}