{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd5f47b7-b89d-56be-85ab-4e4904f126f9","task_key":"default--validation--11441","task_revision_id":"1","upstream_id":"11441","short_description":"HellaSwag validation 11441","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to enjoy coffee houses in amsterdam [title] Understand the liberal way the dutch government deals with drugs. [step] Drugs (apart from alcohol and cigarettes) are illegal in the netherlands. But the dutch make a difference between so called' hard drugs' and' soft drugs'.\",\"endings\":\"['Forcing people to do drugs under the influence is taboo. This means you have to pay extreme or horrific fees and penalties.', \\\"[substeps] Drugs deal with the hard drugs essentially without any promise of rewards. No fixed risk occurs-you don't need them and if they have been serviced, they cannot be made available again.\\\", \\\"[title] Know that heroine, cocaine, meth, crack and the likes, are considered' hard drugs'. [step] These drugs are illegal to produce, to trade in, to traffic, to own and to buy.\\\", \\\"The netherland is known for its drug injections and also, from london to frankfurt, colombia is known for its' soft drugs'. Drugs which are used (like tablets, jellies, alcohol, cigarettes) in these areas often get into fights because of men using them.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}