{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8085f886-aa4d-5cc1-867d-017a5ffadacd","task_key":"default--validation--46865","task_revision_id":"1","upstream_id":"46865","short_description":"HellaSwag validation 46865","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to conduct teleseminars [title] Visit any free conference calling service website and create a free account. [step] Registration is a simple process. Provide your email address, first name and last name and you receive an email notification of your account credentials.\",\"endings\":\"['[title] Read the welcome email and familiarize with the conference controls. [step] Decide how you are going to conduct the conference call.', 'In addition, you should get information like the date, as well as your email address or other account details. After completing this, the registration agent will call the conference, send you a schedule for the conference and assign a time for the conference.', 'These are all hosted by the secretary of state at www.nos.gov. [substeps] Take advantage of these free service sites.', \\\"These activate the telephone conference button on your phone. ( you don't need to live near a telephone line to register your business name.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}