{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c92d8c68-0d34-5a2d-9575-2104fc337d56","task_key":"default--validation--18143","task_revision_id":"1","upstream_id":"18143","short_description":"HellaSwag validation 18143","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to breed dwarf hamsters [title] Pick out male and female dwarf hamsters. [step] If you don't already have a sexually mature male and female hamster, you should find one at the pet store. Check the genitalia to ensure that you are getting a male and a female.\",\"endings\":\"[\\\"If the genitalia isn't visibly red, they are not dwarf hamsters. [substeps] Female dwarf hamsters are docile, quiet animals and shy from humans.\\\", '[substeps] Males are generally considered more sexual than females and dwarf hamsters only coexist with female hamsters and dwarf hamsters. Female hamsters, however, cohabitationers and males all share the same racial background in scientific disciplines.', 'It is pretty tricky to tell the difference. For males, the testicles and anus are far apart.', 'You should also find out if you have a lemur, pelvis, or other fat of any sort. [substeps] Dwarf hamsters have meeled conditions, which can cause the meats to overreact.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}