{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7c0db03d-c71b-5f46-818f-b417774deb9d","task_key":"default--validation--3397","task_revision_id":"1","upstream_id":"3397","short_description":"HellaSwag validation 3397","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to take care of your lawn [title] Water in late evening or early morning. [step] Cool, humid, low-wind conditions distribute water evenly and reduce loss to evaporation. Ideal conditions usually occur between 10 pm and midnight, or between 8 am and 9 am.\",\"endings\":\"['[substeps] Concentrate on the ground, not your soil. When the sun goes down, it makes the surrounding environment and the lawn worse.', 'Avoid using weed-killers during hot summer summer weather. These are often found in backyard lawns.', \\\"[substeps] Some local water departments forbid night watering to prevent waste from sprinklers left on all night. [title] Check your lawn's water needs.\\\", 'No matter what time of day, always water your lawn regularly. The closer your lawn is to the sun, the quicker water can be absorbed into the soil.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}