{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a931b05a-f1aa-5552-9528-71e27abcb9d6","task_key":"default--validation--27015","task_revision_id":"1","upstream_id":"27015","short_description":"HellaSwag validation 27015","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to choose a solar inverter [title] Learn about the commonly used solar inverters. [step] Generally speaking, there are three major types of solar inverters on the market, including grid-tie, off-grid and hybrid inverters. [substeps] Grid-tie inverter: it functions to convert dc to ac, with an ability to synchronize to interface with a utility line.\",\"endings\":\"[\\\"This makes inverters between 4 and 6 gb long. Hybrid inverters: they're similar to grid-ties, except not as large.\\\", 'Hybrid inverters use a dial pad system. The dial pad system converts ac into ac and dc.', 'This works by controlling how much electricity there is coming into your tank. It is available in multiplier mode, as described below.', 'This inverter is designed to transmit your unused electricity to the grid and has no battery. Mttp technology may be equipped in its input circuitry.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}