{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7956ad0d-328a-57ea-b3ba-8eb94bbcb308","task_key":"default--validation--37921","task_revision_id":"1","upstream_id":"37921","short_description":"HellaSwag validation 37921","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get the most amazing legs [title] Make it a habit to walk for thirty to forty-five minutes daily, either morning or evening, whatever time suits you. [step] Try brisk walking or light jogging at one pace. [title] If you find it hard to walk by yourself, ask a friend to come along or take your dog or an ipod with you.\",\"endings\":\"[\\\"[title] Find a comfortable spot to work out. [title] Since our main focus is on our thighs and calves we'll stress more on those areas.\\\", \\\"[title] Look at all the beautiful beauty on the planet. [step] There is no shame in dying yourself white! Make sure you're rich enough to do it.\\\", \\\"[step] If you aren't able to cross the street, try other ways of improving your balance and power through physical challenges. [title] Even if you're not a great person, you can still do great things on your own.\\\", '[step] Try not to get too long strides, as you can get tired faster than if the walk is done slowly. Be careful not to exceed 30 minutes, as light strides often slow down.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}