{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f4a1a8c7-bf55-54ed-a585-5ad9801dcacf","task_key":"default--validation--20112","task_revision_id":"1","upstream_id":"20112","short_description":"HellaSwag validation 20112","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to store iris bulbs [title] Dig up the bulbs carefully using a spade. [step] Make a small hole about 2 to 3 inches (5.1 to 7.6 cm) around the area where your irises are growing. Dig until you reach the bulb, then set your spade aside to continue uncovering the bulb with your hands.\",\"endings\":\"['[substeps] Be careful when digging around the bulb; the water can turn the dirt into a sludge and a dangerous hazard. Aim the spade at the bulb, rather than at the bulbs themselves.', '[title] Transfer the bulbs from the hole into a carrying bag or container. [step] If the bag is too heavy for you to move the bulbs with one hand, use a heavy metal object, such as a rock, to gently lift them out of the bag.', \\\"[substeps] The bulb might have roots growing from it. In that case, be very careful and try to keep as many roots intact as possible while you're removing it from the ground.\\\", '[substeps] Be careful handling the bulb with bare hands, as if you were to pick it off the foliage using your fingers. Avoid digging holes that are wider than two inches (5.1 cm).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}