{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"211715fa-9bd7-56f8-b67d-526adb4c22d8","task_key":"default--validation--45417","task_revision_id":"1","upstream_id":"45417","short_description":"HellaSwag validation 45417","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to leave your hairdresser [title] Give your stylist a second chance. [step] Be specific about what you want done at the beginning of the appointment. If you still don't like it, explain to your stylist what it is that you want and what she/he is not doing.\",\"endings\":\"[\\\"Your stylist will understand all you are asking of her/his and will be able to provide professional advice. It is a good idea to pick another stylist when you are willing to walk away with a two day haircut so it's not as stressful.\\\", 'If you want a certain look bring pictures from a magazine. Most hair stylists are cool about adjusting their work to meet customer specifications.', 'As a result, she/he will be able to help you decide on a hairstyle that you will not tolerate. That being said, let your stylist know what style you feel you would like in order to help ensure it is what you are after.', \\\"[title] Be accepting of your stylist's suggestions. [step] Also, remember that when your hairdresser uses you as a personal stylist, it is your responsibility to accept their suggestions.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}