{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"16ec02c8-9bd6-514b-8a7a-6bb1c96afb4f","task_key":"default--validation--39374","task_revision_id":"1","upstream_id":"39374","short_description":"HellaSwag validation 39374","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to write a retirement letter [title] Discuss your retirement plan with hr. [step] Set up a meeting with a representative of the human resources (hr) department to discuss the details of your retirement plan. Doing this before you write your letter will help you to get an idea about how to shape your letter.\",\"endings\":\"[\\\"[substeps] Ask hr how you can change your plan a few months in advance or how you can update your plan later. Don't simply rely on your old plans to offer changes after a few months have passed.\\\", 'Bring a few financial documents as well as a paper clip so that you can bring the documents in your envelope. [substeps] This is a good time to talk with hr at the appointed meeting.', 'An hr representative is an employee who travels the world. He or she will be able to review your retirement plan and facilitate changes to make it easier to accept.', '[substeps] When you meet with hr, bring any questions you have with you and take notes during the meeting. Jot down important details, as well as details you feel might pertain to your letter.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}