{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9e486cda-becf-5fd4-82d4-e9f3412016fe","task_key":"default--validation--7472","task_revision_id":"1","upstream_id":"7472","short_description":"HellaSwag validation 7472","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to tell if your baby has colic [title] Prepare yourself for intense bouts of crying. [step] A colicky baby wails loudly, and the cries often start without warning and come in waves. You will likely find yourself unable to comfort your baby during these crying fits.\",\"endings\":\"[\\\"As you move toward your baby's enrichment for the first time, you will likely be concerned that he or she is sick or injured. If you ignore the baby's attempts to calm yourself, you might inadvertently injure him or her.\\\", \\\"It's always best to buy a bottle or syringe and prepare yourself beforehand for large (and potentially deadly) bouts of crying. [substeps] During any infant's initial crying session, you need to be prepared to take measures to accommodate their screaming.\\\", 'When these episodes begin, make an effort to contact the parents and speak soothingly to them. Make sure to always keep an ear handy during these bouts of crying.', \\\"[title] Watch your baby's body language. [step] Many babies suffering from colic stretch their legs out or pull their feet up under their bodies, as though trying to ease a cramp.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}