{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5129f7b-0756-5838-b5b2-5eb8a041cb93","task_key":"default--validation--41810","task_revision_id":"1","upstream_id":"41810","short_description":"HellaSwag validation 41810","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to learn a programming language [title] Determine your area of interest. [step] You can start learning with any programming language (though some are definitely \\\" easier \\\" than others), so you'll want to start by asking yourself what it is you want to accomplish by learning a programming language. This will help you determine what type of programming you should pursue, and provide you a good starting point.\",\"endings\":\"['[substeps] Find out what genres are currently popular and most likely bring a lot of questions to your mind. This can help you reflect on your passion for the programming concept that is causing you to be on the spot.', '[title] Write down two or three goals you hope to achieve over the course of a few weeks. [step] Begin with one goal and work backwards towards the next goal.', '[substeps] Your objective is to learn a language that you are good at and enjoy doing. This means no terrible language that is of little recognition or of no intent.', \\\"[substeps] If you want to get into web development, you'll have a whole different set of languages that you'll need to learn as opposed to developing computer programs. Mobile app developing requires a different skillset than machine programming.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}