{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fcf150ce-31e6-5a4d-a675-fa37799a1f28","task_key":"default--validation--30947","task_revision_id":"1","upstream_id":"30947","short_description":"HellaSwag validation 30947","config":"default","split":"validation","body":"{\"activity_label\":\"Triple jump\",\"ctx\":\"Once finishes, she begins doing a triple jump into the pit, she tries again, and then the man interviewing her begins doing the same jump. the two\",\"endings\":\"['try and jump the rope several times before finally pulling it away.', 'come back to the camera and finish talking, she flashes her thumbs up and then leaves.', 'women then take an air kiss as the reporter walks away and returns with a box and begins presenting it to the other woman.', 'men are talking in a room and then the host appears and begins receiving two claps while others ignore them and secure the jump boots.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}