{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e887e777-3373-5fb3-9de8-10a84a66eb82","task_key":"default--validation--15681","task_revision_id":"1","upstream_id":"15681","short_description":"HellaSwag validation 15681","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to preserve fruit [title] Choose ripe, flavourful fruit. [step] No matter what type of fruit you're canning, the flavor and texture will hold up best over time if you use fruit that's perfectly ripe. Discard fruit that's overripe and bruised, and leave out fruit that's not quite ripe yet.\",\"endings\":\"['Pick fresh fruits with a smooth, active appearance to preserve them as well. What you choose depends on your taste and the health of your canning equipment, though fruits of all ages and varieties usually function better when preserved in good form.', '[substeps] This can be any fruit, from wild strawberries to coconuts to melyfruits to peaches and bananas. Fruit that\\\\'s got a flavour, fragrance or taste \\\" natural \\\" is more likely to clog up the filter.', \\\"[title] Process the fruit according to a recipe. [step] Since each type of fruit has different properties, it's a good idea to follow a canning recipe designed just for the fruit you're preserving.\\\", 'The dark and sticky flesh of fruit should be consumed immediately upon being harvested. Fruit that cannot be disposed of is not edible.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}