{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d4e005d0-be02-5d01-8ed9-455655f5c742","task_key":"default--validation--34666","task_revision_id":"1","upstream_id":"34666","short_description":"HellaSwag validation 34666","config":"default","split":"validation","body":"{\"activity_label\":\"Philosophy and Religion\",\"ctx\":\"[header] How to earn fuel rewards [title] Research fuel rewards programs. [step] Many different gas stations, grocery stores, and credit card companies have fuel rewards programs. Do an internet search for programs in your area.\",\"endings\":\"['Look for programs that will be the most convenient for you, such as one through the grocery store or gas station you visit the most, or a program through a credit card you already have. [substeps] Read the fine print carefully so you know how rewards are earned, what the maximum discount is, when the rewards expire, and so forth.', 'Go to your local paper shopping for a corporate fuel service program. Search the program if it is convenient to you.', '[substeps] Check local ranger stations, coffee shops, and grocery stores for fuel rewards programs. See what the prices are for fuel rewards programs.', \\\"Don't rely on the exactly how much you spent on fuel. For example, walmart may train computers or credit cards at the same time.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}