{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f9242324-275c-5b74-9963-ac1956f91da1","task_key":"default--validation--16914","task_revision_id":"1","upstream_id":"16914","short_description":"HellaSwag validation 16914","config":"default","split":"validation","body":"{\"activity_label\":\"Holidays and Traditions\",\"ctx\":\"[header] How to properly prune roses in the summer [title] Use sharp scissor-type bypass pruners. [step] Bypass pruners have sharp, curved, scissors-type blades that overlap each other, while other types like anvil pruners have straight upper blades that cut against flat, lower plates. Although anvil pruners are usually cheaper than bypass ones, they will crush the stems of the roses.\",\"endings\":\"['So avoid using anvil-type pruners and dull pruners to prune roses. [title] Wear gardening gloves that have long cuffs to cover your wrists.', '[substeps] Do not use bypass pruners if you are pruning plants that grow much larger. [title] Try weed control by using the scissor-shaped bypass shears.', \\\"[substeps] Talk to a professional florist if you want to avoid lengthier cuts. Trimming a plant's foliage with bypass pruners can be dangerous.\\\", \\\"Use garden-grade bypass pruners if necessary only when pruning a rose at a time. [substeps] To prune your shrub, don't use shears or scissors.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}