{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36956681-8f71-5573-9c3e-de6653bb5abb","task_key":"default--validation--36666","task_revision_id":"1","upstream_id":"36666","short_description":"HellaSwag validation 36666","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to attain effortless beauty [title] Don't sweat it. [step] Find a physical activity you really enjoy. It does not mean you need to run, do cardio, or do sit-ups.\",\"endings\":\"['This can mean a yoga class, purchase a dvd for a dance routine, or even go bike riding. Sweat is a good thing, working out not only makes you look better, but it also helps you feel better.', 'Just find some exercise you enjoy doing regularly. [substeps] The opposite is often the case with beauty.', 'In fact, being active will do a lot of the work for you and it will make you feel better physically. [substeps] Try going for a nature walk with your dog or running to catch up with a friend.', 'So instead, just stay comfortable and try to do the things you love. Just being yourself and doing it so you are not obsessing over the results in bed or worrying about your looks in the morning.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}