{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d55cf6f0-7619-5d10-ab35-32a2b40fd171","task_key":"default--validation--10028","task_revision_id":"1","upstream_id":"10028","short_description":"HellaSwag validation 10028","config":"default","split":"validation","body":"{\"activity_label\":\"Holidays and Traditions\",\"ctx\":\"[header] How to store an artificial christmas tree [title] Remove the tree-topper from the tree. [step] Get the biggest pieces from the top off first, so you don't feel overwhelmed by it later. [title] Remove all decorations from the tree.\",\"endings\":\"['[step] Remove as much as you can before cleaning the tree up, including the santa claus ornaments. Try to stick to items that are actually real while clearing out any extra ornaments or decorations.', '[step] Remove any old christmas tree ornaments and any leftover removable items. [title] Scrape away holiday cheer, tree decorations, garlands, etc.', \\\"[step] This could include any tinsel and garland (includes popcorn and fruit-bed strands), along with lights (if the tree isn't a pre-lit tree) and decorations. Make sure to completely remove all the tinsel that exists on the tree-down to the last single errant strand that you could miss from year to year.\\\", \\\"[step] Blow drying isn't recommended as it causes condensation. [title] Place the tree on a large tray or a desk, and allow it to air dry for a few days.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}