{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a6466fcc-a397-57a4-8053-0fba76e34ef2","task_key":"default--validation--1810","task_revision_id":"1","upstream_id":"1810","short_description":"HellaSwag validation 1810","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to qualify for food stamps [title] Have a valid social security number. [step] You and everyone in your household must have a valid social security number if order for your household to qualify. [substeps] On an additional note, you and everyone in your household must also be u.s. citizens or u.s. nationals.\",\"endings\":\"['For example, if you belong to a college or university, you should have a valid social security number. A credit card is the most widely available one for u.s.', 'Individuals other than you and the intended recipient of the food stamps will not be considered foreign immigrants. Note that most people in us territories will be recognized and covered by the u.s.', 'When you file a claim you will provide proof of name change or citizenship. A different social security number is generally acceptable in case there are protests or other publicity problems related to the type of transaction.', 'If you are legal alien, you might still qualify as long as certain conditions are still met. See the \\\" special cases \\\" section for more detail.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}