{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"df37ea83-728f-5a8a-b5cb-1458317fbe57","task_key":"default--validation--1068","task_revision_id":"1","upstream_id":"1068","short_description":"HellaSwag validation 1068","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to replace fluorescent lighting [title] Make sure that the power is switched off. [step] Before replacing a fluorescent light, you should make sure the fixture is not powered on. Do not simply use the wall switch to remove power to the fixture.\",\"endings\":\"['Check the appropriate circuit breaker on the back of the light fixture to see if this switch is working. [substeps] If the switch looks broken, or failing to turn off, then try turning off the power supply and replacing the fixture.', 'Removing power to a fluorescent lights fixture causes damage to your electrical system. To be sure that the fixture is dead, remove it from the outlet, turn off the power to the circuit, and assess whether you need to replace it.', \\\"You can use the fuse box to shut off power to the lamp's circuit. Use a voltage tester to verify that there is no power in the fixture.\\\", 'Here are some signs that the power is on : [substeps] There are flickering lights around. These lights indicate that the current is too low and the light is not shining properly.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}