{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2ac9b3e8-9a57-512d-89df-bea4e8ce5039","task_key":"default--validation--13370","task_revision_id":"1","upstream_id":"13370","short_description":"HellaSwag validation 13370","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to help an itching dog [title] Check for fleas. [step] If you see your dog scratching constantly, check its skin for dark brown or black specks. If you are not certain if your dog has fleas, comb it gently with a fine tooth comb, getting very close to the skin.\",\"endings\":\"[\\\"Fleas can build up on bald spots and as a result, your dog's skin may become irritated. Flea bites can also lead to a rash.\\\", 'If it starts scratching, stop when it stops, and check for fleas at the first sign of irritation. Fleas may look like tiny red pinworms, but may be tiny feces or bite marks.', 'These clear specks could indicate fleas. [substeps] If you spot fleas in a stray dog, wait for it to relax before you begin treating it.', 'Tap the comb onto a wet, white paper towel and look for reddish-brown flecks, often called flea dirt. You may also catch fleas as you comb.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}