{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5c341080-a5fc-5dac-9cbe-35153e4fffa1","task_key":"default--validation--34936","task_revision_id":"1","upstream_id":"34936","short_description":"HellaSwag validation 34936","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to write a letter to the irs [title] Format a business letter. [step] Open a blank word processing document and set the font to something readable, such as times new roman 12 point. Remember that business letters are single spaced and use block paragraphing, which means you don't indent with each new paragraph.\",\"endings\":\"['To keep your letters legible, you must set the font to something readable, such as times new roman 12 point. [substeps] The same applies to business or corporate letters.', '[substeps] You should also read the letter over several times to clarify any assumptions. Also begin to identify any subsidiaries at the beginning.', 'Instead, you leave a blank line between paragraphs. [substeps] You can use letterhead if you have it.', 'Instead, you should use a single block font, such as times new roman 11 point. [substeps] The standard format includes one or two paragraphs, each with a comma, one space, and the period.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}