{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1a741c3b-4886-59b9-9618-c6d3ca6617e7","task_key":"default--validation--34072","task_revision_id":"1","upstream_id":"34072","short_description":"HellaSwag validation 34072","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get a haircut you will like [title] Get recommendations. [step] The best way to determine if a hair stylist is a good fit for your needs is to talk to clients who've worked with him or her. That means asking people with haircuts that you admire for stylist recommendations.\",\"endings\":\"[\\\"If a friend or family member always seems to have a really great cut, ask who styles their hair so you can consult the stylist. [substeps] You don't just have to ask people who you know for stylist recommendations.\\\", '[substeps] If possible, talk to people who you respect personally. For example, if you admire someone at work who works with a certain brand, ask to speak to them for a few minutes to find out what they think of those qualities.', 'People who know you and the stylists you know might be able to advise you. [substeps] Ask former hair salon and haircare professionals with hair styles you like and other stylist recommendations you have about getting a stylist.', 'See their feedback to see if a stylist is right for you. [substeps] For instance, talk to john combs if you know this photographer you are talking with.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}