{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"afaac6c5-03c6-5b25-a3bc-54ac00fe34a1","task_key":"default--validation--28840","task_revision_id":"1","upstream_id":"28840","short_description":"HellaSwag validation 28840","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to use himalayan salt lamps [title] Remove contaminants from the air. [step] You may be wondering how a chunk of salt with a light bulb in it can clean and deodorize the air. The answer lies in a process, called hygroscopy.\",\"endings\":\"['In this procedure, a portion of the sodium necessary to form rock salt is covered by a layer of fine shavings to partially deodorize the air. This allows the salt to be passed through the water slowly as it wears through the air.', \\\"Essentially, it uses a device inserted into a strand of the bulb's light. At this point, the light enters into the strand and filters out the methane.\\\", 'Himalayan salt attracts water molecules in the air and absorbs them. These water molecules carry tiny particles of dust, pollen, and smoke with them, which are deposited on the surface of the salt lamp.', 'This procedure uses a microorganisms that filter toxins out of the air and release them into the air. You and your friends can observe the results of the procedure, then remove yourself.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}