{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0b476367-cada-54e5-a0b2-31dfc24db394","task_key":"default--validation--39102","task_revision_id":"1","upstream_id":"39102","short_description":"HellaSwag validation 39102","config":"default","split":"validation","body":"{\"activity_label\":\"Doing crunches\",\"ctx\":\"A woman demonstrates posture and exercise technique while sitting on a floor in work out clothes. a woman in work out clothes\",\"endings\":\"['serves coffee in a coach level room when shown learning exercise in a form conducive to groping out to diamond chain workout.', 'practices pull back the shower nozzle attached to the side of the shower head.', \\\"poses with a goal in her hands demonstrates how to hold the red balloon at a steady pace using her feet and by posing with the balloon's handles, neck, and hips.\\\", 'is sitting on the floor and talking to the camera when she suddenly turns to the side, still talking to the camera but pointing now to her shoulders and waist.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}