{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"993acf52-5451-5f90-8c14-e3400d7319f8","task_key":"default--validation--29396","task_revision_id":"1","upstream_id":"29396","short_description":"HellaSwag validation 29396","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to create a family tree diagram [title] Write down the people you want to include in the diagram. [step] A family tree begins with you, and branches out from there. Start by writing down the names of people in your immediate family, then move to your parents' generation.\",\"endings\":\"['List all of the people in your family as well as the people who share their genes with you, like grandparents, great grandparents, or grandfathers. [title] Include the year of conception in your tree.', \\\"Make sure you don't leave anyone out! Your family tree will be an important piece of family history, so take time to create an accurate diagram. [substeps] Write down your name, your siblings' names, and your parents' names.\\\", 'Begin with these names and work your way backwards from there. [substeps] For example, mary was older than me about 2000 years ago.', \\\"( as you continue to grow up, your family tree can vary a great deal). [substeps] You don't have to stick to the same names every time, but start with someone who actually grew up in the same house, and builds a family tree like yours.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}