{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9e3f2dd4-27e4-5683-b289-3788b0e2dbb7","task_key":"default--validation--39987","task_revision_id":"1","upstream_id":"39987","short_description":"HellaSwag validation 39987","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to babyproof your home [title] Install baby-proof/child-proof products around your home. [step] Products include locks for cabinets, outlet covers, corner guards, and furniture straps. [title] Keep hazardous products away form children.\",\"endings\":\"['[step] Store chemicals, glass, medicines, knives, forks, pots, pans, spoons, spices, and other sharp or hazardous objects away from the reach of children. Instead, put these items in locked cabinets or drawers.', '[step] Dangerous products include adhesives, varnish, and graphite shavings. Inspect your home and ask if there are hazards in plain sight before you entrust your child with it.', '[step] Make sure children are not exposed to potentially hazardous products, like smoke, fire, and poisonous bugs. [substeps] Clothing and clothing may contain pesticides, insect or disease-preventing agents, and homes with bugs in these items may be a breeding ground for insect mites.', '[step] Children are often startled by products with dangerous chemicals in them. [title] Keep your personal safety concerns in mind.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}