{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2332a396-ef2b-504d-b7a6-c665030c9806","task_key":"default--validation--29541","task_revision_id":"1","upstream_id":"29541","short_description":"HellaSwag validation 29541","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to get a second family car on a budget [title] Create a budget. [step] Take a look at the types of cars you may want for your second vehicle and set an overall price range goal toward which you can work. From there, take a look at your current budget and determine how much you can put away each month toward that goal.\",\"endings\":\"['For example, you may decide to spend 20% of your money on groceries and 30% on vacation if you drive the same family. You may then spend another portion of your total on things like vehicle maintenance, utility fees, mortgage payment, and insurance.', '[substeps] For example, if you do not have enough money in your budget to devote to your new car and for this month your fixed costs will be shorter. You may have to come up with a consistent budget in order to meet your fixed expenses.', '[substeps] For example, you may take a look at your social media accounts each month and determine how much you can only put away each month. You may explain to your parents you need to save 5% for childcare costs and 17% for alimony payments.', \\\"[substeps] Don't forget to include costs such as insurance. Get an estimate for the type of car you are working toward so that you know how much you need to keep it insured and running smoothly.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}