{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5ed5e33e-1baf-58b2-a4e3-22eac12f1c67","task_key":"default--validation--15098","task_revision_id":"1","upstream_id":"15098","short_description":"HellaSwag validation 15098","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to sign up for file storage at dropbox and use public folder at dropbox [title] Go to http: // www.dropbox.com it is available for free for windows, mac, linux and mobile devices. [step] Click on setup file to run it on your computer. A welcome screen will appear.\",\"endings\":\"['Click on \\\" install \\\" button to start the installation process. After the dropbox installation will start running on your computer, you will see a progress bar that will show the progress of the installation.', '[title] Confirm the actions that you want to take. [title] Locate your default file type in the box, download file explorer and double click on file explorer.', '[title] Select \\\" file \\\" and then a settings dialog box will appear. [step] Click \\\" file privacy protocol \\\" to enable public folder sharing.', '[title] Click on install file and you are ready to start. [title] Make sure you have the appropriate computer name (vconfig or php), the computers file format (php) and your url if desired, and the folder name for the file to which you have the appropriate installation preferences.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}