{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a792b012-18b0-51fc-ab5b-902b635f7bf2","task_key":"default--validation--24335","task_revision_id":"1","upstream_id":"24335","short_description":"HellaSwag validation 24335","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear white [title] Choose linens and cottons during the warmer months, especially summer. [step] Summer is the perfect time to break out your favorite white pants, skirts, and dresses. The bright sun will give your skin that perfect glow, and it will help make your white clothes sparkle even more.\",\"endings\":\"['You can even try a white cotton or linen blazer during the summer. [substeps] Try something loose and flowing in the summer, such as peasant blouses or maxi dresses.', \\\"Go for linen and cotton for a more conservative look. [substeps] Cotton is a breathable fabric and it's easy to remove without leaving stains.\\\", \\\"[title] Always lay out your clothes and put on your work clothes beforehand. [step] If you're going to be making dinner or running errands, you should always be ready with any outfit you wear.\\\", 'Summer is great for lighter and more subtle washes, so find something slightly similar to tan in materials like cotton or linen. [title] Pick dark colors when choosing sunshorts and dresses.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}