{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6cd335f5-3f44-52ef-86af-79c6a0302e51","task_key":"default--validation--429","task_revision_id":"1","upstream_id":"429","short_description":"HellaSwag validation 429","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make calendula oil [title] Pick or buy calendula. [step] While you can buy dried calendula flowers at a herb store or online, many people making calendula oil make it with flowers they grow themselves. Pick the flowers when they bloom, when these blooms are at their peak, but wait until mid day so that the dew has evaporated from the petals.\",\"endings\":\"['Harvest the dried petals when the flowers are yellow or cloudy and store them in an airtight container. [substeps] You can purchase dried calendula flowers online or at specialty shop.', '[substeps] Calendula oil is made with just the flower of the plant. Pick the flowers right under the bloom, below the base of the flower which holds the petals.', '[substeps] Check the label that is included with the flowers to see how much water the flowers use. You can usually find calendula by visiting your local nursery.', '[substeps] Calendula grows best when cut into individual bundles. Look for dried flowers in recycled materials like plastic containers.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}