{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6bc23c82-2623-5129-9c51-38935833376f","task_key":"default--validation--23928","task_revision_id":"1","upstream_id":"23928","short_description":"HellaSwag validation 23928","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to beat level 165 on candy crush [title] Get rid of any bombs asap. [step] The bombs in level 165 will explode after seven turns if they are not removed. An exploding bomb will immediately end the level, regardless of what else is going on.\",\"endings\":\"['Because bombs are so high-risk, clearing the bombs should be your top priority. [substeps] Bombs are colored, and can be cleared by being included in combos.', '[substeps] Do not use bombs on medium to large amounts of zombies. One splash will stun the largest percentage of your team and deadly the rest.', 'Be careful, as there may not be any escape routes. [title] Use the \\\" magic tea \\\" throwing machine; based on the mass of the bombs thrown by your friends or enemies.', \\\"If you are in the gym, access the main lobby for the level as you approach it. Remember, it's just knocking up a wall just to crush a good chunk of food.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}