{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"69d73173-d403-51c9-94ec-0c20af8602b3","task_key":"default--validation--6183--variant-147fc1e627a4","task_revision_id":"1","upstream_id":"6183--variant-147fc1e627a4","short_description":"HellaSwag validation 6183","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to groom a gerbil [title] Assess your gerbil's coat. [step] The best way to determine if your gerbil needs to be groomed is by examining his coat. If your gerbil's hair appears greasy or oily, it may be time to let him groom himself.\",\"endings\":\"[\\\"It's especially important if your gerbil does not like to groom himself as it can cause damage to his coat, which could lead to fewer future hair follicles. [substeps] Dirty or bald hair caused by disease can make grooming ineffective.\\\", 'Healthy gerbils have a healthy coat that holds the coat in place. However, be aware that this coat change can be caused by various conditions including fleas, ticks, and nonperianal ticks.', \\\"[substeps] Grooming takes up a great deal of a gerbil's day, and along with eating and sleeping, it is one of his main activities. [title] Give your gerbil a dust bath.\\\", 'A dirty coat is because the gerbil is consistently shedding. If your gerbil has greasy fur, he is probably too dry.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}