{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fd26bb3b-79fd-52c0-8e74-e193fdc9853f","task_key":"default--validation--3168","task_revision_id":"1","upstream_id":"3168","short_description":"HellaSwag validation 3168","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to cut a man's hair [title] Ask the man how he wants his hair. [step] In a consultation, find out how short he wants his hair, how uniformly he wants it cut, and how he plans to style his hair afterwards. He may want you to only trim away split ends, but he could also want you to cut everything off.\",\"endings\":\"[\\\"[title] Consider how long he wants his hair to be. [step] Men don't grow his hair to the shoulder length they normally do, so it would be reasonable to cut his hair any length that he likes.\\\", \\\"Ask that he only cut out the hair he really wants, and that you only cut off slightly more than that at some point. [title] Brush your hair when you're finished cutting it.\\\", \\\"[substeps] Once you've narrowed your list, you'll need to decide where you would like his hair to fall. Some men go all out and have a trim every three to four weeks.\\\", \\\"Before you begin to cut, it's important that you ask all of the necessary questions and understand what he would like done with his hair. If you are clear and communicative from the start, you will avoid any potential misunderstandings.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}