{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1e3279ac-3ece-50f8-b5f4-92ccfcfdb8f9","task_key":"default--validation--731","task_revision_id":"1","upstream_id":"731","short_description":"HellaSwag validation 731","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to discipline cats [title] Recognize the complications. [step] Cats can be easy or difficult to train. Dogs are more eager to please their owners, which makes them generally easier to train than cats.\",\"endings\":\"[\\\"Because of this, cats respond to rules differently. If you want to discipline your cat but you don't have many resources to teach them, then you should stop your training early.\\\", '[substeps] Your cat is more likely to obey if it feels like there are rewards for its behavior. Reward is the sign of a desired behavior, not just a reward.', 'However, cats can be trained as long as you understand how cats think and interact with their humans. Training a cat takes a lot of patience, especially since it is so hard to motivate him.', 'It can be far from healthy to train a cat that is getting older and more rambunctious. [substeps] Older cats in their twenties or older can seem more handful-minded, but older cats in their twenties can be more independent and adventurous.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}