{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c1c77dba-a3d0-5bc8-b3e9-43454477d60c","task_key":"default--validation--5977","task_revision_id":"1","upstream_id":"5977","short_description":"HellaSwag validation 5977","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to stop a cat from biting and scratching [title] React calmly. [step] Never hit, shout, chase, or lose your temper with a cat. You will simply terrify the cat and cause it to become nervous and confused.\",\"endings\":\"['[substeps] Learn to calm yourself with breathing techniques. Take deep breaths to reduce stress and anger.', \\\"If you aren't not sure what to do, try to help your cat and find some supplies to help him calm down. Doing this will reduce potential anxiety and minimize the chance of the cat biting.\\\", \\\"[substeps] Never call a cat to you and then punish it. The cat hasn't got a clue why you're responding negatively to him.\\\", 'By simply being calm around the cat, you will prevent it from attacking you. Your first reaction as a calm and non-aggressive cat will be to chase it away from you.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}