{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"741e5789-bd27-501b-af0b-502a5e18e7eb","task_key":"default--validation--33440","task_revision_id":"1","upstream_id":"33440","short_description":"HellaSwag validation 33440","config":"default","split":"validation","body":"{\"activity_label\":\"Playing pool\",\"ctx\":\"It flashes back to the pool table. The man and woman continue talking and the woman laughs at what the man is saying. the man\",\"endings\":\"['is now teaching the woman how to shoot the pool ball while standing behind her.', 'then gestures towards the people sitting in his pool.', \\\"'s shot is showcased and minor mistakes are made.\\\", 'walks towards the camera and faces is talking again.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}