{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"83c5e26f-38bc-5da5-a012-8ba97a326d0e","task_key":"default--validation--15011","task_revision_id":"1","upstream_id":"15011","short_description":"HellaSwag validation 15011","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to make a digital clock [title] Identify an electronics parts supplier. [step] Many suppliers are available on the internet or through electronics supply catalogs. Choose a vendor that offers basic integrated circuits of the generic 74xx and 40xx families, and discrete components, such as resistors and capacitors.\",\"endings\":\"['Follow the directions on the machine kit kit package to complete the programming process. [title] Learn the approximate hour range of your clock.', \\\"[substeps] 80 kilowatt electronics is typical for 40xx and 40xx families. You can also sign up for the retailer's official music retailer website and search for reviews of products in the music manufacturers department.\\\", \\\"You may have to wait several months to establish a valued vendor, for example : [substeps] Check the local government's website. Contact the vendor directly to negotiate a specific connection, then wait 3 to 4 months for an exchange rate.\\\", '[title] Acquire an electronics prototyping breadboard. [step] If the breadboard does not come with clips, which are wires that can be plugged into the breadboard to connect integrated circuits, purchase the required clips.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}