{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"76e87e59-80c2-565c-b6a4-5f8699adaea9","task_key":"default--validation--28579","task_revision_id":"1","upstream_id":"28579","short_description":"HellaSwag validation 28579","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to connect a guitar or bass guitar to a macbook [title] Consider a di box. [step] You'll need some kind of interface or adapter that takes the signal from the guitar and gets it to the computer. A di is a small amplifier which takes the (quiet) signal coming from your instrument and amplifies it before it goes into your macbook.\",\"endings\":\"['Your computer computer would have the di box and macbook connecting to it. [title] Cover the macbook with plastic wrap.', \\\"[substeps] Di box's seats into the instrument and can be switched if you don't already have them. [title] Place your guitar on your lap or directly upon your room's floor.\\\", 'Dis come in either active or passive forms. Most six string guitars are passive by design (no amplification inside the instrument itself requiring a battery) and thus will require an active di.', '[title] Put a lid on the macbook, use a cable or screw in the front), and plug the macbook onto the auxiliary ports. [step] Slip the finder bar on the front of the box into its proper position.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}