{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3260b7b0-de1d-55c4-a1cb-5472dab02c18","task_key":"default--validation--25104","task_revision_id":"1","upstream_id":"25104","short_description":"HellaSwag validation 25104","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to wire a zone valve [title] Familiarize yourself with the instruction manual for your zoning system. [step] Your boiler's owner's manual may also have instructions for zone valve wiring. Look for any diagrams to help you connect valve wiring in the system's control box.\",\"endings\":\"['[substeps] Be sure to read the instruction manual carefully when wiring a stove, as it may tell you which building codes to use or install the system. [title] Connect the pipe line or other pipe line to the valve faucet.', \\\"Many zoning systems have a hud application that helps you to coordinate wiring. [substeps] You could visit the zoning board's website to learn more about circuit controls and certification online or check out any old circuit guides that may be available on the internet.\\\", 'These diagrams may also include diagrams to connect valve stem clamps using flathead wiring. [title] Fill the gasket with smoked resin.', \\\"[substeps] If you do not have your particular zoning system's instruction manual, try looking for it online through the manufacturer's website. [title] Turn off all electricity to your boiler.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}