{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"049068cf-080d-5962-b696-cf58f79f0086","task_key":"default--validation--11944","task_revision_id":"1","upstream_id":"11944","short_description":"HellaSwag validation 11944","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to use torrents [title] Understand the basics of torrents. [step] Torrent files allow you to download virtually any type of file from everyone else that is sharing it. Torrents allow files to be split into many small chunks.\",\"endings\":\"['These chunks are then downloaded from other users who already have that chunk on their computer. Once all of the pieces are downloaded, the file is reconstructed and is ready to use on your computer.', 'This allows files to download in your server at a faster pace so they can be used for a long time. [substeps] 3 users may be able to download in a single session at a time.', 'Torrent files can then be purchased by you in installments for free. [substeps] Torrents are available on bitmap and other websites with an indexing graphic to help you keep track of your torrent cache.', 'To split the file : [substeps] Open a folder using your browser. Type torrent in the search bar to begin accessing the file.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}