{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8abb1c9f-4edd-55f8-8f6a-d3dcced9e264","task_key":"default--validation--36389","task_revision_id":"1","upstream_id":"36389","short_description":"HellaSwag validation 36389","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to exercise an option [title] Compare the price of the underlying stock to your strike price. [step] When you exercise your option, you buy (call) or sell (put) the underlying stock at the price stated in the contract. If your options have value relative to the actual stock price, you are \\\" in the money.\",\"endings\":\"['\\\" [substeps] A call option allows you to buy stock at the stated strike price. You\\\\'ll make money if the stock is trading at a higher price than your stock price, because you can buy shares at your lower strike price.', '\\\" in other words, you sell the stock without paying a commission. In exchange for wasting money in your initial purchase, you lose an opponent\\\\'s stock.', '\\\" [title] Learn the definition of \\\" strike \\\" in limited use. [step] Strike does not refer to anything over a set price.', '\\\" if the underlying stock price is lower than the collective market value, you are \\\" in the money. \\\" [substeps] In the united states, the general profit of an option is $37 , 750.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}