{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e25f2e46-d155-53d9-8b8f-b029ee204930","task_key":"default--validation--25852","task_revision_id":"1","upstream_id":"25852","short_description":"HellaSwag validation 25852","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to avoid predators [title] Avoid dangerous places. [step] While a predator can attack from anywhere, you'll give yourself an advantage if you stick to the beaten path. At night, try to avoid walking by yourself.\",\"endings\":\"[\\\"Change the streets or turn on your lights every 10 minutes to deter predators. See a predator's presence as the first sign of his or her intent to attack.\\\", 'Stay in a tree bed, around a water source, or simply standing near a fire. [substeps] Avoid areas that are near roads, trees, utility lines, and large, open bodies of water, such as lakes, streams, and oceans.', \\\"You'll do well to avoid places like : [substeps] High-crime areas isolated side streets dark alleyways [title] Be aware of your surroundings. [step] A predator might try to approach if you look like you are lost or confused.\\\", \\\"This may make stalking dangerous, but if you find yourself in a dangerous place, eventually you'll come face to face with the predator. [substeps] Carry a weapon or hand your keys to a friend if you can.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}