{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"17836de1-5f4b-5d53-816c-a92cdcdc5867","task_key":"default--validation--1874--variant-3a438d80fd8f","task_revision_id":"1","upstream_id":"1874--variant-3a438d80fd8f","short_description":"HellaSwag validation 1874","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to choose the right credit card [title] Determine your credit score. [step] Most of the best cards require good to excellent credit. Your fico score (the term refers to the company that created the formula for determining the score) shows lenders how much of a credit risk you are.\",\"endings\":\"['The score goes from 350 to 850, and the higher your score is, the better. If you live in the united states, you are entitled by law to one free credit report per year from each of the three major credit monitoring companies, experian, equifax, and transunion.', 'However, you also want to know your credit score. This is not always an exact science, but in fact you must know your credit score.', 'Thus, paying off your minimum credit scores is a low-cost option. However, that statement is quite unlikely to be the same as paying off your current credit card debt.', 'Lenders need to be truly concerned about your credit score. Given that their credit score is not generally very high, credit card companies prefer to evaluate both your score and the quality of the credit.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}