{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2dc30ce5-2702-5b8b-bce1-ca24446a2b66","task_key":"default--validation--27951","task_revision_id":"1","upstream_id":"27951","short_description":"HellaSwag validation 27951","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to organize a diaper bag [title] Put the diapers in first. [step] Start by placing the diapers in the bag, stacked side by side. Have the middle of the diaper facing up so they are easy to pull out and open when you need them.\",\"endings\":\"[\\\"If possible, put the diapers on lengthwise, so they don't get too flat on top of each other and leave the later parts partially open so that they can be disposed of later. [substeps] The diaper bag makes loading much easier if you have a stand-alone diaper bag.\\\", '[substeps] Put the diapers in one side of the diaper bag. Depending on how big your diaper bag is, you may only be able to fit four to five diapers.', '[substeps] Add extra cloth diapers first, if needed, if you get more than one diaper. You can even place the diapers together in separate compartments.', '[title] Tie off the loose ends of the strings. [step] Each diaper should have a loose end.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}