{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"74608fcd-7942-5f99-92e9-a74484c6f2c1","task_key":"default--validation--30508","task_revision_id":"1","upstream_id":"30508","short_description":"HellaSwag validation 30508","config":"default","split":"validation","body":"{\"activity_label\":\"Philosophy and Religion\",\"ctx\":\"[header] How to get in touch with your inner pisces [title] Ditch the order. [step] If you have a job or calling in life that requires a lot of order for you, it can be stifling. You need to whip up a little chaos.\",\"endings\":\"['Start with a messy desk or brighter clothes. Perhaps this will surprise others but you must let the real you shine through.', \\\"[substeps] Your server needs to be sufficiently chaotic for them to stand up for themselves. If they can't get it (or are being too lazy), they're easy targets.\\\", \\\"Don't just' run' everywhere. Show up at the office and lie low, dim the lights a bit, make faces and drop the secret to the boss.\\\", \\\"[substeps] Think of some way to make things easier for yourself by doing only what needs to be done. If it's your job, you can try working at it.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}