{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c31b9e03-9f92-52f2-a14e-03a4c9e2c770","task_key":"default--validation--50015","task_revision_id":"1","upstream_id":"50015","short_description":"HellaSwag validation 50015","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to calculate how much money you need to retire [title] Determine basic living expenses. [step] An important first step is to determine the amount necessary to cover basic living expenses each year. There are different perspectives on how much this will be.\",\"endings\":\"['Educate yourself about how much your monthly living expenses will actually be. [substeps] Calculate your annual expenses by multiplying your spending money by your monthly contributions.', \\\"Paying the conventional living allowance or about $500 dollars per month should cover relatively minor expenses, including mowing the lawn. To realize the safety issues you will need to write a few charts describing some minor expenses that you might not even consider doing if all of the above haven't been considered.\\\", \\\"[substeps] Some experts believe you should simply calculate your current expenses. Then, expect you'll need about the same amount to live on once you retire.\\\", 'Some sources suggest as much as five times the market cost per year. Others suggest over eight times the actual cost for basic living.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}