{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d7737467-7056-54c1-a0ff-1d2cac5a0fea","task_key":"default--validation--7346","task_revision_id":"1","upstream_id":"7346","short_description":"HellaSwag validation 7346","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to write a breach of contract letter [title] Create a letterhead. [step] Your letterhead can be as simple or as elaborate as you wish. At a minimum, it should include your name, address, phone number, and e-mail address.\",\"endings\":\"[\\\"[substeps] If you use microsoft word, you can download free letterhead templates on microsoft's website. [title] State the delivery method below the letterhead.\\\", '[substeps] Use a formal font like arial, times new roman, or arial 12 point, as the colon ensures freshness and gives your letter a polished, professional appearance. You can also add typos or smaller forms.', '[substeps] Make sure the letterhead space is only enough so that you can clearly see the letter and write it with italics. Notice any underlines that might require an additional letterhead.', 'Include the full legal name of the person or business you are suing. [substeps] Print out your letterhead on thick, professionally colored paper with white and black text.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}