{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a1182ae4-e3c9-5e99-9c10-4542fee22dcd","task_key":"default--validation--4165","task_revision_id":"1","upstream_id":"4165","short_description":"HellaSwag validation 4165","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to read cat food labels [title] Look for net weight. [step] In the u.s., many brands abbreviate this as \\\" net wt \\\" and list it at the bottom of the front of the bag or can. This is the weight of the contents of the package.\",\"endings\":\"['[substeps] Net is used to indicate a container weight. Some companies, such as amazon, sell these under different names.', 'For example, a net weight of 3 oz. ( 85 g) means the package contains three ounces/85 grams of food.', 'Net weight can be anywhere from 0.57.5-1.119 kg. For instance, if you check a 12 ounce package of litter or pee tube from a local pet store, it should go in the pedometer (m-h) range.', \\\"[substeps] Net weight varies by brand, so you'll probably need to go to a local supermarket, department store, or even pharmacy for more options. American express may be accurate with net weight, although not all stores are.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}