{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"71d1a887-ba9d-5169-a242-df38f95971cc","task_key":"default--validation--26600","task_revision_id":"1","upstream_id":"26600","short_description":"HellaSwag validation 26600","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to add variable products to woocommerce [title] Log in to your wordpress dashboard. [step] This assumes that you already have the woocommerce plugin installed. You then need to have a look at your left hand menu and click on products] add product [title] Fill in the title for your product as you would do in a normal blog post.\",\"endings\":\"['[step] Also add a description in the text body. [title] Scroll down the page, then by product data you have to select variable product.', '[step] Next, type in \\\" productivity \\\" without the quotation marks. Click and then put your own stylus somewhere near the title.', '[title] Click on the name of the product that appears in your blog. [step] It should be as specific as possible because it needs to focus the market to that particular product (p).', '[step] Be sure to delete any quotation marks that make it difficult to put in your new title. [substeps] Name the product when you add it and how long that will be.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}