{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"144f97e6-8582-5858-8576-d60d5abd30d7","task_key":"default--validation--35300","task_revision_id":"1","upstream_id":"35300","short_description":"HellaSwag validation 35300","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to sync your google calendar from your computer to your blackberry [title] Start the process by accessing google online. [step] If you don't have an account already you will need a google account to make a calendar. [title] Search for google mobile in the search field.\",\"endings\":\"['[title] Select \\\" google phone \\\" in the \\\" settings \\\" field. [step] After this, the start menu will appear to the right.', '[title] Enter the google mobile webpage and there you will see options for cell phone types. [step] I have a blackberry so i will be teaching how to sync your calendar to that phone type, but it is possible to sync a google calendar to other phone types with internet access.', '[step] For example, you can search \\\" google today! \\\" to search for google lunch restaurants in your area. Every time a menu appears for a different category, use the search tool or the keypad to search for it.', '[step] Sometimes this field will be already loaded, but if this is the case then you may have not set it up yet. Otherwise, wait and see if this new field opens up, or if it does you have the option to turn on google control.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}