{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e167a862-5f87-580a-931b-17d6cd4f139e","task_key":"default--validation--38341","task_revision_id":"1","upstream_id":"38341","short_description":"HellaSwag validation 38341","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to unlock your blackberry [title] Get your blackberry's imei number. [step] This unique identifier is required by either your carrier or the third-party unlock provider in order to get your unlock code. To retrieve your imei number, follow the appropriate steps for your device : [substeps] Blackberry 10-go to settings, then select advanced.\",\"endings\":\"['For security purposes, make sure to bypass security glitches and trackpoint editing. Blackberry 11-go to settings, then tap on security in the upper right.', 'For portable devices, this should be on the kitchen or living room of your device. Blackberry 7-go to menu, then select on device, then select cellular.', 'Tap hardware from the dropdown menu that appears. Your imei number will be listed in the window.', \\\"By default, if you don't see the unlock code displayed, then it's likely the device has been locked and connected to your cellular network. Mac 8-go to options-then scroll down and tap cellular network settings.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}