{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c3de1950-d18c-585a-ac1b-3b8be96f46c5","task_key":"default--validation--7764","task_revision_id":"1","upstream_id":"7764","short_description":"HellaSwag validation 7764","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to plan your bachelorette party drinking games [title] Don't hold back with this on. [step] The key of this drinking game is to get as personal as possible. This bachelorette game is really fun to play when everyone is tipsy! [title] Each of the players take a turn asking a specific chosen player a personal question.\",\"endings\":\"['[step] You can create the game completely on facebook and in one very simple by asking the player who asked the guest to \\\" have fun! \\\", and do how the guests respond. [title] Have a couple of gals help you.', '[title] Do this for two rounds rotating clockwise. [title] The player being questioned, answers \\\" not in my pants \\\" to every question asked and must not laugh, smile, or grin until their turn is over.', \\\"[step] They can ask several of the same questions, or they can gauge each person's sobriety. You may find yourself cheering along with the others, or listening to your reflection in the mirror.\\\", \\\"[step] If a player doesn't know an answer or a dish of strippers you will attract stronger loser-girls and girls around this stage. [title] Decide on a date for the bachelorette party.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}