{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b2deb66a-b5ec-5728-b6a0-6ff17824ae0b","task_key":"default--validation--20010","task_revision_id":"1","upstream_id":"20010","short_description":"HellaSwag validation 20010","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"She showed an orange bottle, read the label and squirt the orange content on her palm, showed the cream on the camera, then rub the cream all over her face. she\",\"endings\":\"['bend down and rinse her face, when her face is visible on the camera her face is clear.', 'then took a glass of water and then smear it on her face while holding onto a tube of mouthwash and then hold onto a spray bottle in her hand after she puts it down.', 'took it off, put it on her breasts, rub the cream on her belly button and the back part of her legs.', 'kissed the kies, then used the black scrub and the cream all over her face.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}