{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d9d30c6e-5b63-5b09-9b0a-6b74bbb524b2","task_key":"default--validation--11504","task_revision_id":"1","upstream_id":"11504","short_description":"HellaSwag validation 11504","config":"default","split":"validation","body":"{\"activity_label\":\"Shaving legs\",\"ctx\":\"A woman is spreading shaving cream on another woman's legs. She puts her hands under water and begins to shave the woman's legs for her. she\",\"endings\":\"['is then shown shaving her tattoo and applying the cream or just relaxing.', 'then uses an one inch razor to shave her legs.', 'starts from bottom to top of the leg and continues that motion.', \\\"rubs soap onto the women's legs and rubs them with her hands.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}