{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"76fb5547-de35-54fb-8691-de2e01ed99a9","task_key":"default--validation--10246--variant-c4c0915683b5","task_revision_id":"1","upstream_id":"10246--variant-c4c0915683b5","short_description":"HellaSwag validation 10246","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to diagnose and treat gingivitis in cats [title] Look at their gums. [step] To check your cat's gums, sit down with the cat. Tip their head back, and part their lips with a finger so that you can see their gums.\",\"endings\":\"['[substeps] You can also gently slide your finger in between their upper teeth. Your finger should be just under the gum line, and you should not see any visible blood vessels.', \\\"Your cat's gums are upper and lower than those of a normal cat. [substeps] If your cat's gums are whiter than normal, it likely has gingivitis.\\\", 'Then, look for white gums, a pink gum line, and stripes or markings. These are all signs of gingivitis.', 'Typically, gingivitis will cause redness and swelling around the gum line. You may also notice some plaque build-up on their teeth.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}