{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"30b2a511-92c7-520f-aa55-07b121a80732","task_key":"default--validation--10269","task_revision_id":"1","upstream_id":"10269","short_description":"HellaSwag validation 10269","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to lower electricity bills in the summer [title] Cut down on energy leaks. [step] This includes turning off lights and other electronics. When you leave a room, shut off the light behind you.\",\"endings\":\"['This also removes unnecessary energy from overhead electrical outlets. [substeps] If you are handling potentially hazardous electrical materials, bring a flashlight with you.', \\\"Don't leave the television on for an extended period of time, as the light from the screen may affect the ability of the night light to generate electricity. Excess energy can spoil the efficiency of your home, and might even affect the quality of your finances.\\\", \\\"Unplug electronics that aren't being used, such as cell phone chargers, small appliances like toasters, or power strips that provide power for many appliances. [substeps] Big-screen tvs, dvd players, digital photo frames, and other appliances use more energy than you realize.\\\", 'Fluorescent light bulbs and power strips in bathrooms often clog up bathroom water pipes and the toilet. Turn off the water at least once a day and check for leaks, especially in pipes that are unresponsive to electricity.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}