{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b36108f6-ea9a-52e0-9c63-1120eeb3eeaa","task_key":"default--validation--6325","task_revision_id":"1","upstream_id":"6325","short_description":"HellaSwag validation 6325","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to dispute a credit card charge [title] Identify the unauthorized charges. [step] An unauthorized charge is any purchase on your credit card that you did not make or authorize. Take out your credit card statement and identify the dates, amount, and merchant of any unauthorized charges.\",\"endings\":\"['Review this statement to see who made the purchase and why. [substeps] If you have several unauthorized purchases, write down all receipts and billing history with your statements.', 'State both the name, address, and current account number on your credit card statement. The merchant of your card will have to confirm or deny the charge.', 'The unauthorized charges will appear on your card receipt. You should also have all receipts for purchases done on your credit card and any payment that was made on your personal credit card.', 'You will need this information to report to your credit card company. [substeps] If your credit card statement does not arrive on time, call your credit card company.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}