{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d23338e-8a96-5868-9b2a-c68854605f8f","task_key":"default--validation--44597--variant-82ba059aec61","task_revision_id":"1","upstream_id":"44597--variant-82ba059aec61","short_description":"HellaSwag validation 44597","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to get a free domain [title] Start an email account. [step] If for whatever reason you do not already have an email account in your name, now's the time to start one. You will need an email address to register a domain.\",\"endings\":\"['An email address is one that you automatically create when you start a new new business. [substeps] For example, if your first domain is myspace, this is your primary email address.', '[substeps] Start up an email server by entering your email address and the url of the domain name. Sign in to your account and explain to the administrator the responsibilities involved.', \\\"You will also need a password to register a domain. [substeps] Each domain's username is a different, but the basic grammar, spelling, and punctuation are the same.\\\", \\\"The options for free emails are numerous, although gmail is arguably the best for this purpose. [substeps] Password and email security is always important, but it's made even more vital if you're associating your email with a new domain name.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}