{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f6591cd7-e9fe-502f-b2f2-8b4c372bfef3","task_key":"default--validation--25125","task_revision_id":"1","upstream_id":"25125","short_description":"HellaSwag validation 25125","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to configure an spf record for your domain [title] Identify your domain. [step] Per the wizard, you will be asked to enter the name of your domain for which you would like to create an spf record (i.e. contoso.com) in the field box. It is important to know your domain name in order to begin this process.\",\"endings\":\"[\\\"Obviously, if you don't know the name of your domain, how can you verify it? [title] Display published dns records. [step] Next, the wizard would check dns for information about the domain.\\\", '[substeps] You will get a notification when this is successfully done for you. You will need this notification from spf to get started.', '[substeps] You will be given a template or application that can tell you what your domain name is. Copy the template or application to a new document or in a file folder.', '[title] Look at the spfs logo and the \\\" personal show \\\" thumbnail. [step] Uncheck the box or \\\" external show \\\" thumbnail, then enter all the spfs.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}