{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"85521881-5fbd-5585-b4f7-c3c115630ac4","task_key":"default--validation--4453","task_revision_id":"1","upstream_id":"4453","short_description":"HellaSwag validation 4453","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to book a blood donation appointment in canada [title] Open up a browser of your choice and visit the canadian blood services website, then click the log in button in the top right. [title] Enter your login details and press log in. [title] Select \\\" my appointments \\\" on the left menu of the website.\",\"endings\":\"['[title] Click the \\\" save \\\" button next to \\\" change your date \\\" to create an account and pay for your consultations. [step] You can also add some payment points for these appointments if you wish.', '[step] This will bring up a window titled \\\" deals \\\". [title] Click the checkbox labelled \\\" have a blood donation done in canada.', '[title] Select the payment link next to the \\\" receive your blood \\\" option, which is listed next to the \\\" v \\\". [title] After confirming the payment, a confirmation message will appear on the screen for the recipient.', '[title] Click on \\\" book an appointment \\\". [title] Enter an address and a date range, then click on find clinics to search for the nearest donation clinic.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}