{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d188b037-7de9-54b2-bf16-14a2883595ce","task_key":"default--validation--1324","task_revision_id":"1","upstream_id":"1324","short_description":"HellaSwag validation 1324","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to coordinate a fashion show [title] Decide on a theme. [step] All fashion shows are centered around a general theme. [substeps] A theme could be fall fashion, beach fashion, lingerie, or costumes.\",\"endings\":\"[\\\"Or it could be a secondary theme like a junior prom or new year's eve. [title] Tell the directors of your fashion show.\\\", 'Alternatively, the theme could focus on a particular color or fabric. The music, lighting, and decorations will have to go along with this theme.', 'If you believe fashion is important to each audience, brainstorm topics of interest to your audience. A theme could include asian cuisine, outdoor skydiving, obedience classes, etc.', 'Your choice could be a superhero themed fashion show, boys sports professional, or photography. [title] Pick april 1st or july 15th, which are theme days.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}