{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6aaf5f1e-933a-5c4c-8669-ccd812e5f381","task_key":"default--validation--14475","task_revision_id":"1","upstream_id":"14475","short_description":"HellaSwag validation 14475","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to earn growtokens on growtopia [title] Learn about crazy jim. [step] Crazy jim is an npc in growtopia that can be called by a telephone by dialing 12345. He offers you a daily quest everyday as well as three life goals.\",\"endings\":\"[\\\"To earn growtokens from him, you'll have to complete all three goals as well as the daily quest for that day. For every life goal you complete, your awesomeness level raises by 1 %.\\\", '[title] Obtain the exoder00 dungeon level for 260 there are only four monsters in growtopia that can outspan 240 at 5/4 point. [title] Use the dungeon level as your way to level 365 in the game.', 'Once you get in the growtopia, you will soon have a reward on your taxes, which will earn you a range of growtokens. [substeps] The adult growtopia can sometimes earn lots of money for a lot of money.', \\\"[title] Provide a list of the fun games your friends are playing. [step] They don't have to be actual game items to earn growtokens but it's nice to get some of the fun games that are available for you to play.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}