{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"00444290-0f13-5503-89fe-a0c6cbf2b40b","task_key":"default--validation--10901","task_revision_id":"1","upstream_id":"10901","short_description":"HellaSwag validation 10901","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to create a models' essentials' kit [title] Carry a compact mirror. [step] Invest in a pretty compact mirror and you will always be able to check your appearance. [title] Have some breath mints.\",\"endings\":\"['[step] ( first aid kit ) [title] Special kit essentials kit checklist to have. [title] Know that you will need the following supplies : [substeps] Classic weapon, claymores, map headphone, ps2 stick-screws ready-bought rucksack or carry-in camera extra clothing.', '[step] Use the mints in the car and do a brushing change. If you are going out in the car, bring a pocket release and a pack of mints.', '[step] Mints can work really well for a variety of reasons. [substeps] A regular bag will be enough to transport all your supplies, staples, school supplies and work essentials in one place while your class supplies and accounting materials are scattered across the floor.', \\\"[step] Hummus may have seemed like a really good idea at lunchtime but if you'd rather not breathe it all over a potential agent, then some mints may be a welcome addition to your' must-have' kit. [title] Be sure to have tissues or wet wipes.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}