{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5df0231f-e7a2-5ed5-b6a0-d58ed1d2d597","task_key":"default--validation--39053","task_revision_id":"1","upstream_id":"39053","short_description":"HellaSwag validation 39053","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to change a diaper in a car [title] Get out of the car and open the boot (trunk). [title] Clear space in the boot for baby and check for sharp things. [title] Lay down a nappy (diaper) changing mat.\",\"endings\":\"[\\\"[step] If your car has one, lay it down at a height that allows you to change your diaper faster while also preventing you from being touched by your baby and yanking off the end of the cover. Push down your baby's ankles slightly to open the boot.\\\", \\\"[step] If you don't have a mat you can use a carrier bag by carefully ripping the sides to make it big enough. [title] Lay new nappy down in middle of mat.\\\", '[title] Strip down some of the seat and ditch the seat belt (both soft and soft seats usually allow the baby to twist on and stick to them more easily). [title] Reverse the bike (gear up (as when you reversed position) and clean inside of the device carefully (oil, brake and engine cleaning before putting back in the car).', '[title] Ease baby down and put baby on the mat. [title] Gently pick up baby and put him on the mat [title] Find clothing suitable for you and your baby.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}