{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"013e9f06-9750-58c4-a459-d97f5ddd4936","task_key":"default--validation--42319","task_revision_id":"1","upstream_id":"42319","short_description":"HellaSwag validation 42319","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to install an ice maker [title] Locate the closest cold water source. [step] To install an ice maker you will have to tap into the closest-located cold water pipe. Find this cold water source, which may be found under a sink, below the floor, or in a wall.\",\"endings\":\"['[substeps] A cold-water pipe may seem to be taped or taped diagonally, but most electric ovens will wrap the new water source around this particular pipe. [title] Look for positive owner-talk signs.', '[substeps] Usually, you can visit the local icebreaker testing center to find a remote location. Ice removal companies that offer ice run kits are also good sources of electricity, water, and electricity.', 'If you need to drill through the floor or wall to reach the pipe, it is best to hire a professional who will know how to avoid wires, ducts, or other plumbing. [substeps] If you do not know where the pipe is located, ask your contractor, a former resident, or landlord.', \\\"Tap the pipe in so that it is warmer than your house's cold water source; it is important to note that moisture can deteriorate the system which might result in faulty training or failure to use the stove. [substeps] Ice makers come with associated heating settings for different types of system.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}