{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e37d8045-1636-584e-985f-c5e193b1e4ff","task_key":"default--validation--12891","task_revision_id":"1","upstream_id":"12891","short_description":"HellaSwag validation 12891","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to buy a mattress [title] Visit mattress websites to see what's offered. [step] If you haven't purchased a mattress in a while, it's good to see what options are available before going to a store. [substeps] Check prices online to see what you feel is reasonable based on what is being offered.\",\"endings\":\"['Look for online deals, but check with your local grocery and drug stores if you have questions about how much that can cost. You can say something like \\\" hi, my favorite mattress will cost $800 \\\".', 'Mattress brands are often coming out with new styles of mattresses including ones that offer adjustable firmness levels and temperatures. Decide how high-tech you want your mattress to be, as some of these may only be available at specialty stores or online.', 'There are websites like handylist.com that have more than a handful of suppliers. [title] Get a stylish, functional mattress.', \\\"Even if the prices are lower than what you need, you don't want to get stuck buying a mattress that is under warranty. Sites like ebay often offer discounts on mattresses of different prices.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}