{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b8ab46e1-a4de-55c4-8687-ed70cd5a942a","task_key":"default--validation--45642","task_revision_id":"1","upstream_id":"45642","short_description":"HellaSwag validation 45642","config":"default","split":"validation","body":"{\"activity_label\":\"Wrapping presents\",\"ctx\":\"Bright pink french cancan candy wrap is shown. The materials, tools and and how to make it are demonstrated step by step. after\",\"endings\":\"[', a white text appears with dark black words read \\\" basketball santa claus \\\" and black computer graphics.', ', plants of different kinds are shown.', ', ingredients are added such as baking soda.', 'wrapping the ribbon is used to tie the ends and ruffles are made on each end.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}