{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c1f97155-b300-5bf6-8a46-233b8c076f7c","task_key":"default--validation--28396","task_revision_id":"1","upstream_id":"28396","short_description":"HellaSwag validation 28396","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to bond with a stray cat [title] Safety first. [step] Don't compromise your own safety. This includes not going into any area that may be dangerous, such as a construction site, an abandoned mining area or a dilapidated building.\",\"endings\":\"[\\\"Moreover, don't put yourself in a precarious position to try to reach the cat. Make sure you are on a solid surface that won't fall out from under you or cause you to slip or fall.\\\", \\\"Instead, gather supplies and tools you won't likely need as you get a stray cat. [substeps] Grab paints, gauze (or a few strips of black gauze), roughly 2 tablespoons (15 ml) of fabric, or masking tape.\\\", 'You might also be insulted if your stray cat bumps into a neighbor or another old friend. Stray cats will not behave well if people who encounter them leave them alone.', \\\"Safety first means safety without being outside in tightly enclosed spaces. If the stray cat is new to the home, or seems scared or panicked, or continues to keep circling its territory, there is a good chance it's being tracked.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}