{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fb2be30e-4341-5009-9539-a7a97893ae32","task_key":"default--validation--16728","task_revision_id":"1","upstream_id":"16728","short_description":"HellaSwag validation 16728","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to pronounce oregon [title] Listen to someone say \\\" oregon \\\". [step] Visit the state, if possible, or speak with someone who has spent time there. Listen to an audio pronunciation sample : [substeps] Https: // en. Wikipedia. Org/wiki/oregon https: // www.youtube.com/watch? V = wtapsr6knt4 [title] Emphasize the first syllable.\",\"endings\":\"['[step] Start this question with the sound of \\\" oregon \\\" followed by the new word after it. Note if your listener is unsure of the correct first syllable.', '[step] Pay attention to how the first syllable is pronounced throughout the word and plant the second syllable in between the words so it sounds like a noun. The easiest and most common way is to say \\\" oregon.', \\\"[step] Make sure you include the first syllable or highest syllable of the word, but don't attach it to the original word. This will indicate to the listener what the original phrase means.\\\", '[step] The correct pronunciation uses three syllables: \\\" or \\\", \\\" ih \\\" and \\\" gun \\\". The first syllable is long but the last two syllables are very short.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}