{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce96d795-37d6-5f87-97a6-bd2349f97628","task_key":"default--validation--20419","task_revision_id":"1","upstream_id":"20419","short_description":"HellaSwag validation 20419","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to get rid of tiny ants [title] Find the ants' entry areas. [step] Ants come into your home for two main reasons: to find food to bring back to the colony, and to seek shelter from cold weather conditions. Ants could enter your home from many different areas, so be sure to check around window cracks, door entryways, lighting fixtures, and other possible entrances.\",\"endings\":\"[\\\"This is especially important if the ants you're dealing with are migrating from outside. [substeps] If you can't find where ants are immigrating from, check a nearby garage, attic, chimney, or other exposed area.\\\", 'If you see a trail of ants in your home, follow them to see where they are entering from. [substeps] If you find specific entry points where ants are coming into your home, focus your repellant and deterrent efforts on these areas.', '[substeps] Give extra attention to the entrance to your home or apartment when searching for food. If you frequently use doors in the dining room, backyard, nursery, or hallways, these are probably your best choices.', '[substeps] See if your house or home has cement tiles, concrete, or metal flooring, especially. Small ants prefer these materials to hard surfaces.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}