{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"34b41dc6-0fc2-5286-a967-6548af1a2464","task_key":"default--validation--44774","task_revision_id":"1","upstream_id":"44774","short_description":"HellaSwag validation 44774","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to buy and sell used video games [title] Locate a retailer near you that buys and sells used games. [step] Used game retailers include gamestop, play n trade, and best buy. [title] Find the used video game section for your chosen console.\",\"endings\":\"['[step] There are many little tab to the right of the retailer and you may see little gear labeled \\\" buying and selling \\\". [title] Choose a game that will suit your budget and interests.', \\\"[step] Usually, you'll only have to find it once if the game is in great condition, so don't get discouraged if the console is far from being in good condition. You'll need to open your games quickly so that you can access them quickly.\\\", '[step] After finding the retailer, search online for the game you would like to buy. Once you find the game, enter your search query, and click on \\\" check game download details.', \\\"[step] You'll most commonly find sections for playstation 3, playstation 2, playstation portable, xbox 360, xbox, wii, nintendo ds, nintendo 3ds, gamecube, and pc. [title] Choose the title that you want and bring the game's case to the cashier.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}