{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d631f6ea-46e6-5647-b221-64702a511feb","task_key":"default--validation--19992","task_revision_id":"1","upstream_id":"19992","short_description":"HellaSwag validation 19992","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to draft an agreement for the sale of goods [title] Format your document. [step] Set the font to a legible size and style. Although times new roman 12 point is fairly standard, you can pick anything that is readable.\",\"endings\":\"[\\\"[substeps] If you intend to use this contract over and over, then you can create a template. Insert blank lines where you can write in information that will change with each contract, such as the buyer's name, delivery date, price, etc.\\\", 'If the length and font adjustments are not workable, you should include the postal code twice the standard length and font size as well as the addresses of the people selling to you. [substeps] Write them alongside the date you purchased the goods and description of the product.', '[title] Start by showing how the buyer must understand the terms of an agreement. [step] Keep all receipts and other information that is needed to draft an agreement.', 'This is best used when you want to make sure your signature will set the tone for your agreement. [substeps] Avoid bolding, italicizing, or underlining your words.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}