{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e78eb164-7ae8-553b-ab24-838069d78009","task_key":"default--validation--47291","task_revision_id":"1","upstream_id":"47291","short_description":"HellaSwag validation 47291","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write a letter for proof of income [title] Begin by listing your contact information at the top of the page. [step] This should include the name, address, telephone number and email address of the entity responsible for preparing the verification of income, in this case you. [substeps] If you are self-employed, you should include your business name and information.\",\"endings\":\"['If you do not know this information, list your social security number on the appropriate line. Another option is to place your corporate name on the first line of your letter.', 'You should also list other information such as your job. [title] Familiarize yourself with the type of verification you will need.', 'If you have social security numbers, then include these, too. If you are a homeowner, then include a business phone number as well.', 'This information should be positioned at the top left of the page. Separate it from the next section by inserting one blank line below it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}