{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2bea2781-d12a-5452-b7fb-851fcf037065","task_key":"default--validation--18495","task_revision_id":"1","upstream_id":"18495","short_description":"HellaSwag validation 18495","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to short sale your home [title] Determine the value of your property. [step] This is usually a pretty simple step. You may have an idea, based on your town's appraisal for real estate tax purposes.\",\"endings\":\"['If not, ask a friend or family member to take notes so they can refer to what you have done and look at comparable properties. [substeps] Look up the value of your home on the internet and either look at the value on demand or have a friend or family member pay you a a fee to compare your property with it.', 'However, the best way to estimate value is to contact a real estate broker and ask for an opinion of the value. The broker will study your home and compare it with recent sales of other properties in your area.', 'If you decide to shorten your sale even more, you will need to know and make sure it is worth buying. This note will help you quickly determine what is worth purchasing.', 'Or it may simply be what you want to buy, so long as the value is fairly close to what you want. If your home is being sold, then it may be worth the financial effort if you want to make a good profit.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}