{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c64ff70-7ba5-5584-afe0-7f26286a243b","task_key":"default--validation--38221","task_revision_id":"1","upstream_id":"38221","short_description":"HellaSwag validation 38221","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to download fantasy games [title] Find a fantasy games that seems interesting. [step] This can be done by using search engines like google, that let's you type in a string of words and then google finds whatever you're looking for. You can access google by typing \\\" \\\" in the url bar at the top of the web browser that you downloaded, and then press enter.\",\"endings\":\"[\\\"A page that has dedicated to fantasy games will come up. [title] Open the' becoming a fantasy wii' game page.\\\", 'An example of keywords that can be used to find fantasy games is \\\" fantasy \\\" and \\\" games \\\", but experiment with different keywords for varying results. Sometimes, not every fantasy games is the perfect one for everyone.', 'This brings up a list of games available online, some of which are far more popular than others. [title] Visit a nintendo/apple website for a list of games to download.', '[substeps] You can browse across most fantasy games by \\\" character \\\" or \\\" plot \\\" until you find one that appeals to you. Let\\\\'s say you\\\\'re looking for a fictional fantasy game that focuses on industrial use.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}