{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"68bbc5f0-496a-540f-b2e1-e1e3b9860d05","task_key":"default--validation--34179","task_revision_id":"1","upstream_id":"34179","short_description":"HellaSwag validation 34179","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to spot and treat ovarian cysts in gerbils [title] Observe your gerbil's body shape. [step] Look for changes in your gerbil's body, particularly in the abdominal area. An ovarian cyst may protrude from your gerbil's side, or from its abdomen, making it look pregnant.\",\"endings\":\"['Note if your female gerbil is starting to look bell or pear-shaped, which is a strong indication an ovarian cyst. [substeps] A swollen abdomen could also be a tumor or pregnancy.', 'In some cases, this cyst may move up in size and length, meaning that its body has gained an extra size. A messenger cyst is less prominent in female gerbils, but it may be apparent during breeding.', \\\"This can be a sign of an infection, poor birth control, or, worse, a sign of a disease. Check your gerbil's abdomen to see if it has any lumps, bumps, or small bumps.\\\", 'Your gerbil may have elongated pelvic organs, making it appear fat. Pelvic organs can also appear as bumps.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}