{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e034132e-6392-562f-aa2b-f1ac57ccfd08","task_key":"default--validation--25335","task_revision_id":"1","upstream_id":"25335","short_description":"HellaSwag validation 25335","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to organize bridal shower games [title] The first kiss fun game : [step] This is a fun game and can be a great icebreaker for the guests who do not know each other. You can start this game by asking the bride-to-be to tell the story of the first time she was kissed by her finance'. The other guests can contribute as well.\",\"endings\":\"[\\\"After they've told the story, have the bride-to-be give her a kiss and greet her in a friendly way. [title] Press the invitation into place and the bride is given the option of exchanging signals.\\\", 'To make it fun you can have a panel of guests who will rate the best kiss story. The one who gets the highest points turns out to be the winner.', \\\"Your friend's favorite foods will be the things her parents cook. Next, ask the bride-to-be to tell the story about the first time she was kissed by her finance'.\\\", \\\"Obviously, this is the safest choice for women who are are young when they aren't in a college class. Be sure to tell the story in a straightforward way and not look to the wedding cake.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}