{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"58cc7067-6785-547c-bd88-98e184bc52a9","task_key":"default--validation--5508","task_revision_id":"1","upstream_id":"5508","short_description":"HellaSwag validation 5508","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to cut baby hair [title] Make sure that your baby is in a good mood. [step] Cranky, fussy babies will move around a lot, which can make cutting difficult. A happy baby may still squirm, but not to the extent of flailing arms and legs.\",\"endings\":\"['[substeps] If your baby moves too much, they may struggle to stay upright. Feeding them, spinning, getting on all fours, and even holding them too close can make your baby move in a different direction and upset someone or lose themselves.', \\\"[title] Cut away the excess hair on the side of the head. [step] Your newborn baby's head should look round and raised when it is in its crib or sling.\\\", \\\"[title] Begin the cutting procedure fairly early on. [step] It may seem like it might be easier as you're getting your baby out of the bath, but make sure that you make it to the allotted time and then prepare for next time.\\\", 'More importantly, you want your baby to associate haircuts with good memories-not memories of being hungry, tired, or just cranky. [substeps] Make sure that your baby is fed and rested.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}