{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96043b2c-3cb6-51fc-9013-5715f5c4a82e","task_key":"default--validation--30229","task_revision_id":"1","upstream_id":"30229","short_description":"HellaSwag validation 30229","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress for work [title] Dress business formal if you work with customers in a professional setting, such as a bank or office. [step] Business formal usually means wearing a suit, tie, dress slacks, and dress shoes. [substeps] Wear clothes that fit.\",\"endings\":\"[\\\"Investing in appropriate attire can be a good way to cut down on the amount of time you spend milling about. Find shoes that fit you well and conform to your workplace's culture.\\\", 'A business casual outfit is best for the workplace in a professional setting than for a night on the town. Something like khakis can work just as well as a suit with a tie.', 'You might shop in jeans or a nice shirt for business casual. On the other hand, plain button-down blouses are a good option for business casual.', 'Clothes that are too tight will be too uncomfortable to work in. Clothes that are too loose will look sloppy and unprofessional.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}