{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"62dc10b8-c45e-57f4-b446-3f788c56ee83","task_key":"default--validation--29378","task_revision_id":"1","upstream_id":"29378","short_description":"HellaSwag validation 29378","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to soothe a baby who has colic [title] Swaddle your baby. [step] They may not like the process, but the results can be amazing. Swaddling is believed to recreate the coziness of the womb, and may enhance the calming impact of other soothing steps.\",\"endings\":\"['One way to do this is to swaddle a baby when they are pregnant. They are best swaddled during the birthing process, when full of warmth from their mother, and when the warmth also feels soothing.', '[substeps] Swaddling may help the baby get into a more relaxed mood. Choose swaddling clothing, as they may soothe the baby more fluidly than swaddling on their own.', \\\"[substeps] Make sure to swaddle your baby long term. Blankets are easily dispensed, and babies don't need as much stuff to soothe them.\\\", '[substeps] Once swaddled, turn baby on their side and gently jiggle them. Shush them loudly-make sure they can hear you over their own screaming.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}