{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4070f335-a398-5278-ae64-9de0a1e744d6","task_key":"default--validation--6181","task_revision_id":"1","upstream_id":"6181","short_description":"HellaSwag validation 6181","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"Both of the girls are laughing and joking with one another until they start to wash their face and clean it with green rags. after\",\"endings\":\"[', the girl takes a rain squeezer and washes her face that is clean and little by little.', 'rinsing their face, they both dry their face off with a towel that is hanging down from the door and come back to the camera to show pure faces.', ', the person on the right cleans their face with a pink toothbrush and also cleans their other face.', 'they have washed the potatoes the two legs turns to lather the rag and the bread cut out of the tub, followed by the pieces of proper version of what had been put in.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}