{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e02bbdc5-d5b6-5cad-bd7c-46d646e7674b","task_key":"default--validation--19298","task_revision_id":"1","upstream_id":"19298","short_description":"HellaSwag validation 19298","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to plan your retirement [title] Determine your planned retirement age. [step] The age at which you will retire has a large impact on your retirement planning. While it is impossible to know, especially early on, when exactly you will retire, planning for a certain age can help guide your decisions.\",\"endings\":\"['For social security purposes, your retirement age can be either be after or before your \\\" full \\\" retirement age. However, you will not receive any benefits until age 62.', '[substeps] For example, even some elderly individuals may prefer retiring at the age of 60. For those of whom a disabled person is a family member, the u.s.', '[substeps] Begin by choosing a prospective retirement date. For instance, you might decide to retire in mid-30.', '[substeps] For the purposes of this article, the age at which you will retire and the date of retirement may differ. For example, older people tend to retire more frequently, while younger people tend to retire earlier.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}