{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cf7965f6-c88d-52e4-af38-96f5619ace65","task_key":"default--validation--13919","task_revision_id":"1","upstream_id":"13919","short_description":"HellaSwag validation 13919","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to make a home safe for toddlers [title] Use cabinet and drawer child latches to keep little hands out of poison and sharp items. [step] Keep heavy items locked away or well out of their reach. If oven doors are not lockable, buy a latch to keep them shut unless you want to watch your child at all times when baking.\",\"endings\":\"[\\\"[title] Use childproof latches for household repairs. [step] They can be safely locked, or were given, because they' stay safe' at the end of a hallway or adjoining small room.\\\", '[title] If you can, leave some items accessible to toddlers that are safe. [step] They can bang on pans or sort your plastic storage containers while you cook or clean.', '[title] Refrain from leaving your baby inside for long periods of time. [step] Children with small hands and easily handled hands can become allergic and can even choke on poisonous foods and vegetables.', 'Lids are very fragile and we should avoid turning them on or checking the lock. [title] Use holes big enough to allow ventilation, light from the oven, and a childproof bulb for the door.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}