{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d9e39775-12ac-5832-aee3-a012ed61554f","task_key":"default--validation--48888","task_revision_id":"1","upstream_id":"48888","short_description":"HellaSwag validation 48888","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to teach your kid adding and subtracting [title] Have them recite 1-10, and then 10-20. [step] Start by saying \\\" 1 \\\" and then have your child repeat after you. Count up to 10 together.\",\"endings\":\"['Give them at least 1-9 times the total. [title] Ask them to subtract 3/4 of their equivalent to 10.', 'Do this at least two times so your child can get used to hearing the numbers out loud. This will also help them learn the ordering of the numbers.', \\\"When they're done repeating, use three or four times per day for each repetition. As they become proficient, they'll be able to do very intricate memorization.\\\", \\\"The better they understand 1-10, the more they'll start to love reciting along with you. [title] Describe the counting skills you'll perform.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}