{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"867e2a32-2d4a-5608-a3a4-a2fd42bccef5","task_key":"default--validation--28064--variant-f9e7010bc87a","task_revision_id":"1","upstream_id":"28064--variant-f9e7010bc87a","short_description":"HellaSwag validation 28064","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to choose a marriage counselor [title] Search for marriage counselors. [step] The first step in finding a marriage counselor is to see which counselors practice in your area. You can look up therapists in your local phone book, by searching online, or by using a reputable therapist database, such as psychology today's find a therapist page.\",\"endings\":\"['[substeps] Online databases let you search by area, by specialty, and by accepted insurance plans. You can also ask your primary care physician for recommendations/referrals.', 'Within the search field, gather the list of therapists you already know or look in your local phone book. You will also want to do some internet searching to find more.', '[substeps] Your choice of care provider could vary depending on your particular situation. If you are in a crisis state, you may need to find a marriage counselor in the same state.', 'All the options you can use to find the counselor are extensive and comprehensive, so you can also look online and print off web pages where you can research counselors and check their credentials. [substeps] Look online for marriage counselors in your area.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}