{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d1e63bf3-811f-5c62-a7a6-43c6c90e3a33","task_key":"default--validation--23318","task_revision_id":"1","upstream_id":"23318","short_description":"HellaSwag validation 23318","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to change a paypal password [title] Go to the paypal login page. [step] Open your web browser and navigate to the paypal login page. [substeps] You can also request a password reset from the paypal mobile app.\",\"endings\":\"['Open this tab and get the link to your downloaded paypal account by clicking the black arrow beside the page name. Navigate to, select, and click the link, then click \\\" request a password reset.', 'Select a button on the upper toolbar beneath the login page. [title] Click the \\\" change password \\\" button on the page.', '[title] Open the \\\" basic password reset/update \\\" box. [step] To make sure that you are using the correct password, you\\\\'ll need to log in with the default login credentials as a normal user.', 'You cannot change your password in the mobile app, however. [title] Click \\\" having trouble logging in? [step] \\\" this link is below the email and password fields.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}