{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e26edde-389f-5a79-8593-75972a7aa230","task_key":"default--validation--22596","task_revision_id":"1","upstream_id":"22596","short_description":"HellaSwag validation 22596","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to calculate the apr on a credit card [title] Read through your credit card agreement. [step] You will receive the agreement when you get your credit card in the mail. A copy of the agreement will also be sent with your monthly statement.\",\"endings\":\"['[substeps] The agreement does not have to be written in your name. In many cases, it will be written by you.', '[substeps] If it has more than one entry, that entry is the ptk you spent that month on your credit card credit card. If you are not satisfied with the only entry, get a new one and spend a little more money to replace it.', 'Read it carefully and determine whether you agree to everything laid out in it. [substeps] If you agree to the terms and conditions in the agreement, you will be able to dispute or cancel the agreement.', \\\"You'll also have access to the information online. The agreement has all of the factors used to calculate your annual percentage rate (apr).\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}