{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"74f87ed0-0a5a-5e6b-9c8a-39dfd16a8ed3","task_key":"default--validation--24315","task_revision_id":"1","upstream_id":"24315","short_description":"HellaSwag validation 24315","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to be preppy, girly, and feminine [title] Develop some good manners. [step] Good manners are a necessity to being feminine, and it's just good to have them. Feminine girls act properly in all situations.\",\"endings\":\"[\\\"Once you have gained some good manners, you can build up your assertiveness again. [substeps] Always wear your best clothes, don't wear funny clothes, and don't walk high heels.\\\", \\\"You never know when you'll need to walk up to a guy or turn up someone's hair. Remain respectful and are friendly with boys who speak to you.\\\", 'They know how to speak, act, write, eat, and be at all times. And they do it all with poise and grace.', 'Learn how to say please and thank you, know how to thank a boy, and communicate properly. Remember not to talk rudely, swear, or take a backward foot.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}