{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c97b8b50-ae4b-54db-b165-129bf3ad46fb","task_key":"default--validation--33529","task_revision_id":"1","upstream_id":"33529","short_description":"HellaSwag validation 33529","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write a payment reminder [title] Format your letter. [step] Your letter should be set up like a. Set the font to something readable-usually times new roman 12 point works for most readers.\",\"endings\":\"['Make sure the font goes at least 6-12 characters lower than the rest of the letter. [substeps] You may also consider having your contact information on the envelope for your immediate address to access as a reminder.', \\\"[substeps] You can do this by lining the front with lines facing each other and writing the recipient's name, address, and other identifying details near the top. [title] First, pull all the relevant documentation together.\\\", \\\"[substeps] Remember to include the date that you are sending the letter. If you don't use company letterhead, then you need to include your business address at the top.\\\", 'If your letter is more personal, set it up like a business letter to keep the recipient informed of the presentation. [substeps] If you are writing a good thank you letter to a friend or colleague, set the font to 4280.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}