{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"55617f32-13b8-54c5-bb5a-60b2ef7fba12","task_key":"default--validation--44322","task_revision_id":"1","upstream_id":"44322","short_description":"HellaSwag validation 44322","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to research genealogy (family history ) [title] Interview family members. [step] Talk to as many family members as you can, especially older members of the family, such as grandparents, who would be happy to give you any information or family history that they know of. [substeps] Consider recording your talks with family members, especially older family members.\",\"endings\":\"[\\\"If the family members have enjoyed spending time with them, you may mention this to others in the family. For example, ask people on the court's website about their pregnancies, stock your mail collection, and personal history.\\\", 'This will make it easier for you to contact them to discuss alternative approaches. For example, you could collect and share a file of these discussions in writing.', \\\"Ask where they were raised, where they were employed, and if there were any family salaries and other health conditions that interests them. It's best to ask questions about family history, as the family's family members will be less likely to be biased against you.\\\", 'These oral records can be kept on your computer for ease of reference and verification of exactly what was said. [title] Talk first to your oldest family member, like your great-grandmother.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}