{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f7851631-d9f4-57f4-abbb-a3246a9af28f","task_key":"default--validation--12147","task_revision_id":"1","upstream_id":"12147","short_description":"HellaSwag validation 12147","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to hide treats for your cat [title] Place treats under furniture. [step] Cats tend to enjoy hiding under furniture like beds, tables, sofas, and bureaus. Place treats in these spots for your cat to find the next time it crawls into its regular hiding spot.\",\"endings\":\"[\\\"This will keep your cat out of reach of the treats when it's about to steal something from you. [substeps] Put old newspapers, old books, magazines, and even plastic toys under the furniture.\\\", '[substeps] You may try scattering a few treats under different items of furniture in your home for your cat to find. [title] Put treats behind drapes or curtains.', '[substeps] If your cat is lazy during the day you may put it in the hiding place on a rug or cushion instead, or you could tuck it in bed when you are trying to feed it. Place the treats in a sticky, colored box or opaque glass container covered in a sticky material like cellophane to create a temporary hide out.', 'This will give the cat a better vantage and allow it to see the treats without it being alerted. [title] Remove furniture once your cat is not coming.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}