{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"98ca6b30-d0a6-534d-a59d-3afe329a9c59","task_key":"default--validation--45174","task_revision_id":"1","upstream_id":"45174","short_description":"HellaSwag validation 45174","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to treat fractures in cats [title] Watch your cat walk. [step] Fractures can affect your cat's ability to walk. For example, if she has fractured a leg bone, she may hold the injured leg up in the air and hop around on the other legs.\",\"endings\":\"['This motion, however, causes the pieces of bone to break. Your cat might faint or rock from side to side as she is in this position.', 'Your cat may also accidentally fall onto her knee and injure herself. [substeps] If your cat is around a particular post-game veterinarian, you should try to steer her toward that vet.', 'If she keeps yelping and jumping around, there may be an underlying emotional issue that means she has been injured. [substeps] Cats with fractures on their feet may otherwise straighten and shift their feet often, which can lead to limping or lightheadedness.', 'Or, if she can put weight on the injured leg, she may walk with a limp. If your cat has fractures in multiple legs, she would likely be unable to walk at all.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}