{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d4ed029f-563a-5911-967e-b1b39660b34d","task_key":"default--validation--12269","task_revision_id":"1","upstream_id":"12269","short_description":"HellaSwag validation 12269","config":"default","split":"validation","body":"{\"activity_label\":\"Clean and jerk\",\"ctx\":\"Three people are in the gym, one of the males is kneeling down and jerking a bar while the other is giving a massage to a woman. the male with the bar\",\"endings\":\"['is doing a sit up on the floor by the women, he reaches for the woman.', 'hit his hand on the back a bull and after the punch, the other man started to read the book.', 'gently talks to the other one, then the male with the weights started to do the same thing with her buttocks.', 'lifts the weights, drops it and then jumps in joy that he has lifted the weight, drops his belt and walks off.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}