{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a8a1c529-0ce5-5541-97ec-9baa607c4e47","task_key":"default--validation--28470","task_revision_id":"1","upstream_id":"28470","short_description":"HellaSwag validation 28470","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to buy a refurbished laptop [title] Determine the features you want in a laptop. [step] You will want to buy a refurbished laptop that can meet your personal needs. For example, if you plan on using your laptop primarily for traveling, you may want to verify that the computer can efficiently connect to wireless internet.\",\"endings\":\"[\\\"If you're looking at refurbished laptops for cash, you will want a laptop that has basic features and graphics cards like processor, keyboard, modem, etc. [title] Calculate the features you want in a laptop.\\\", \\\"You may also want to consider purchasing a refurbished laptop that can connect to a wifi connection as well. [substeps] Examine the laptop's features to see if they are a good fit for you.\\\", '[substeps] Read system requirements on the packaging of software you use. If you plan on using specific software, you will want to buy a refurbished laptop with the system requirements that can efficiently handle your software.', '[substeps] If you prefer to use your laptop for business, you may want a laptop with the capability to connect to wireless internet. However, refurbished laptops may need some maintenance.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}