{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36e57c52-4c55-5cac-b7b8-76e2f159042a","task_key":"default--validation--21301","task_revision_id":"1","upstream_id":"21301","short_description":"HellaSwag validation 21301","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to start a baby registry [title] Work on your baby registry while you are pregnant. [step] Creating a baby registry can be a deceptively long process and may take a few revisions, so it is best to start early. [substeps] Most expectant moms start around twelve weeks, and add to their registry as they find out the gender of the baby.\",\"endings\":\"['You may want to take your baby registry to a facilitator to make the process easier. This can be your doctor or midwife.', 'One good rule of thumb is to starting at twelve weeks before the due date and extending your time between that date and then three or four weeks after. It may take a few weeks after baby steps, in order to start a new registry.', 'Most expectant mothers include different or more than a particular baby gender in their registry. Never duplicate a baby gender before you start the registry process.', 'Aim to have your registry completed by the time you are six months pregnant. Take into account when your baby shower is being held, and give guests a good opportunity to access your registry beforehand.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}