{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"176d455b-a62c-5e33-8b4a-aecd6c9e4925","task_key":"default--validation--45888","task_revision_id":"1","upstream_id":"45888","short_description":"HellaSwag validation 45888","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to keep a friend [title] Loosen up. [step] Don't hold on to grudges and give your friends the benefit of the doubt. Don't assume that your friend is trying to spite you or doesn't care about your relationship.\",\"endings\":\"['The first step towards managing your relationship is relaxed. [substeps] Breathing deeply and slowly works as well, but in doing so is not as simple as regulating your breathing.', 'When they do something wrong, give them a hard time. Though you may not know why they did it, you will be able to find patterns.', 'The key to successfully maintaining a friendship is accepting that each person has flaws. [substeps] Remember that some friends have their strengths, but other people have weaknesses too.', \\\"Consider that your friends have their own lives and that their schedules may change on the spur of the moment. [substeps] Give your friend time to explain herself or let her know directly how you're feeling.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}