{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"41caf27d-785a-5f92-a9e8-57cf4ca5a06c","task_key":"default--validation--37026","task_revision_id":"1","upstream_id":"37026","short_description":"HellaSwag validation 37026","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to report a runaway child [title] Write down the child's name and date of birth. [step] Consult any identification cards, such as a passport or school id, to get the child's first, middle, and last name as well as their date of birth, if necessary. [substeps] Book bags are a good place to look for a school id or driver's license.\",\"endings\":\"[\\\"[title] Consult medical records for the child's most recent height and weight. [step] Locate the visit summary from the child's last pediatrician appointment, if possible, to get the latest height and weight for the child.\\\", \\\"If the teacher expects students to provide identification, be sure to do that. In some places, you may want to search by zip code (if you're unsure about where your child lives) or.\\\", \\\"If your child's name or date of birth is also located on a government-issued id, the name will be at the bottom of the document. Don't use a federal or state government issued id.\\\", 'This does not need to be a legal item. Dress to the nines and enforce a biometric license.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}