{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c0bb00cf-1983-5ffb-bf5a-27eb3d7648ad","task_key":"default--validation--44327","task_revision_id":"1","upstream_id":"44327","short_description":"HellaSwag validation 44327","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to mate anacondas [title] Get a male and a female anaconda. [step] Make sure both anacondas are at least three to four years in age, as this is when they will be sexually mature. Check that both snakes are healthy and have a good body weight.\",\"endings\":\"['[substeps] Before you purchase snakes, be aware of all of your local laws. It is illegal to own dangerous and exotic animals like anacondas in certain places, or you are required to have a special license.', '[substeps] Anacondas have wider stomachs and thicker throats than males. Female anacondas usually have thicker chests instead of smaller ones.', 'If they do not have the same body weight, they should not mate. [title] Get a male and female anacondas.', 'Female reptiles mature around four months, while male reptiles mature around twelve. It does not matter what gender you are (or theirs), but most female anacondas are at least four to six years old.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}