{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1970e87d-1d11-5be2-bebc-ff95441e8c05","task_key":"default--validation--38912","task_revision_id":"1","upstream_id":"38912","short_description":"HellaSwag validation 38912","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be comfortable wearing high heels when you're tall [title] Find the right heel height. [step] If you don't have much experience with wearing high heels, choose a lower heel to start. Start small, with a one inch heel, and continue increasing height as you become more comfortable walking.\",\"endings\":\"[\\\"Once you've mastered the one inch heel, try a one and a half inch heel, and keep increasing your heel height by half of an inch. [substeps] If you want to try different kinds of heels without breaking the bank, try purchasing different types of shoes with heels that can be worn with different outfits.\\\", \\\"[substeps] Once you master the basics of high heels, you can move up to having them measured to your height. Once you're comfortable with the basics, you can work on having them measured more frequently.\\\", \\\"[substeps] If you're feeling less confident about your height, consider different styles of heels. A slight heel is fine as long as it's the correct height.\\\", \\\"For example, you might try a hi heel shoe while standing on the balls of your feet. [substeps] When you pick a high heel, look at how it sits on your ankles while you're walking, as you might find it too high.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}