{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4e6ef333-3f10-5f04-9839-140581f66170","task_key":"default--validation--39554","task_revision_id":"1","upstream_id":"39554","short_description":"HellaSwag validation 39554","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to cook yucca [title] Trim the yucca flowers. [step] Take 1 full cluster of yucca flowers from the top of a yucca plant and cut the stems off of each flower. Use scissors to snip away the green base from each flower.\",\"endings\":\"['Place the yucca flowers on a cutting board and cut a strip from each flower. [substeps] The strips should be about 1 inch (2.5 cm) in diameter and should be cut so that they can fit on your cutting board.', 'Make sure you cut each flower off of the root end so there are none left behind. [substeps] It can be daunting to cut the flowers if they still have their roots.', 'Pull the long central threads (the pistils and stamens) out and discard them. [title] Rinse and soak the flowers for 10 minutes.', 'Remove any blooms or leaves from each flower, as well as any roots. [substeps] Be careful when trimming the flowers in order to still have usable stems and leaves.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}