{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd9a5c59-ee82-5951-8340-ebb7ebbb3286","task_key":"default--validation--19288","task_revision_id":"1","upstream_id":"19288","short_description":"HellaSwag validation 19288","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to decorate a large living room [title] Take a quiz online to find your style. [step] Go online and simply search \\\" decor style quizzes \\\" in order to get some direction. These quizzes will determine your personal style by having you choose between different rooms, accessories, and pieces of furniture.\",\"endings\":\"['Some quizzes are fairly simple, but others ask you to find items to incorporate. They might also ask you to try color palettes.', '[title] Work with your sibling to create a breakfast table. [step] Time is of the essence when sitting down to get this setup, so your sibling could probably happily clock out.', \\\"[title] Pick decorations based on guest's tastes. [step] When decorating a large living room, it's important to choose decorations that your guests like.\\\", 'As you take the quiz, reflect on whether or not you could see yourself relaxing and/or entertaining guests in each space. Some styles that these quizzes may determine you have include : [substeps] Happy modern warm industrial eclectic collector classic glam [title] Look on pinterest and in magazines for inspiration.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}