{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"62ea98b6-b884-5fe3-b4b6-5ed01dc3e568","task_key":"default--validation--12408--variant-8a1629b28130","task_revision_id":"1","upstream_id":"12408--variant-8a1629b28130","short_description":"HellaSwag validation 12408","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to avoid being mugged [title] Avoid listening to music. [step] When walking in public, you should avoid listening to music on a cell phone or other personal listening device. The music could distract you from various dangers, and someone could take advantage of you if you're distracted by listening to your music.\",\"endings\":\"[\\\"[substeps] The best way to avoid being mugged is to listen to music you enjoy and are comfortable with. Anything that's on cds or others also doesn't count.\\\", \\\"[title] Don't play with your smart phone. [step] While it may seem like a normal thing to do, playing with or checking texts on your smartphone could distract you.\\\", \\\"You don't want to hear music while you're walking in traffic or when other people are around. Some of the things you should not listen to on a cell phone include : [substeps] Other distractions that could tempt you to listen to music while walking.\\\", 'You should also avoid going to work unless you are (unfortunately) completely prepared. [substeps] Earbuds are not typically a necessity, as they are only a temporary item that you do not want to steal.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}