{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9670010b-4f6d-5d68-a7a4-e3c6406c2c36","task_key":"default--validation--27673","task_revision_id":"1","upstream_id":"27673","short_description":"HellaSwag validation 27673","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to measure your financial health [title] Review your current housing expenses. [step] This amount would include rent or mortgage payments. In the latter regard, you should also include any homeowner's insurance and property taxes.\",\"endings\":\"['[title] Evaluate your current debt-to-income ratio. [step] To calculate your debt-to-income ratio, multiply your monthly debt-to-income ratio by 8.', '[title] Check if your annual costs were applicable for your home. [step] Annuities count toward your outstanding debt.', '[title] Obtain the total value of your vehicle. [step] This is your total total annual income, and it should be discussed in the following steps.', 'A healthy monthly payback amount in comparison to your gross income would be less that 28 percent. Should your monthly payback amounts exceed this percentage, you should consider increasing your income or decreasing your monthly payments via refinancing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}