{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ac44a985-abd7-52de-b34a-606a2dca2f43","task_key":"default--validation--2756","task_revision_id":"1","upstream_id":"2756","short_description":"HellaSwag validation 2756","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to deal with a mare in heat [title] Learn about a mare's reproductive cycle. [step] Mares are seasonally polyestrous, meaning their reproductive cycles occur at certain times of the year. Typically, a mare's cycling season will last from april through september.\",\"endings\":\"[\\\"Keep a record of the cycle so you can check on the frequency during the mating season. [title] Pay attention to the foal's age.\\\", \\\"[substeps] For example, mare in heat cycles vary in frequency. It isn't uncommon for a mare in her first period to be vulvarian.\\\", \\\"During these months, each cycle will last 18 to 23 days and the mare will be in heat for about 5 of those days. [substeps] A mare's cycling may begin as early as march and end as late as october or november.\\\", 'Mares are prone to reproductive cycles, particularly during this time, as they are only interested in glands in the vulva. [substeps] If you have a stallion mare, you are likely to notice large changes in her body.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}