{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b364acb0-ac1b-58fb-bfb1-0664c4e6b56b","task_key":"default--validation--39726","task_revision_id":"1","upstream_id":"39726","short_description":"HellaSwag validation 39726","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to design a family tree [title] Determine how much detail you need for your family tree. [step] Before you begin, ask yourself how detailed you want the family tree to be. If you're making a family tree for a class at school, you might only need basic information going back a few generations.\",\"endings\":\"[\\\"If you're making a section for a holiday, you might need to design the entire tree to fit your local area. [title] Consider what other holidays you might want included in the tree.\\\", 'Another idea is to ask yourself if your family will be alive still. You want to make sure your family tree can fill your current needs and provide more details than you may otherwise need.', \\\"If you're making a family tree for a specific person or company, maybe there's room for some extra details. [substeps] If you make a lot of family trees, or make large groups of people, you probably will need large details.\\\", \\\"If you're making a family tree that can be shared electronically, you might want to be as detailed as possible. [substeps] Keeping the purpose of the tree in mind can prevent you from feeling overwhelmed as you find and sort through a lot of information.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}