{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1052f5b3-fb97-52a0-b65e-eed44f183a40","task_key":"default--validation--19829--variant-fad788985c19","task_revision_id":"1","upstream_id":"19829--variant-fad788985c19","short_description":"HellaSwag validation 19829","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to keep a cat from waking you up [title] Understand why your cat disturbs you at night. [step] Most cats have basic, obvious reasons for being up at night. Your cat is most likely bored or hungry, or she may need you to clean the litter box.\",\"endings\":\"['[substeps] Your cat is quite often left inside the house all day while you are at work or school with little interaction. She sleeps all day and may be bored at night with no one to play with.', \\\"Always think about the reasons behind your cat's scratching. [substeps] If your cat prefers to sit down, but you can't find your cat's favorite food or a toy, maybe the reason is due to her desire to be quiet during the day.\\\", 'If your cat barks and whines, these kinds of problems can point to the fact that she is trying to get you to clean the box. Beware, however, if your cat wakes you up, she is likely getting food before you go to bed.', \\\"If your cat is disrupting your sleep by waking you in the middle of the night, get up slowly and quietly and avoid startling or waking the cat. [title] Decide what you'll do if your cat decides not to wake you up.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}