{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ed2dca81-8667-5e2c-ae3d-aa8c9bbf932e","task_key":"default--validation--45251","task_revision_id":"1","upstream_id":"45251","short_description":"HellaSwag validation 45251","config":"default","split":"validation","body":"{\"activity_label\":\"Calf roping\",\"ctx\":\"One of the men starts to chase after a bull on his horse. He gets off of his horse and brings the bull down and starts walking back to his horse. he\",\"endings\":\"['helps the bull up off of the ground and gives it a trophy, then the bull changes his tactics and starts chasing another bull off of the land while another woman walks around the area.', 'then ends up dismounting and waving to the camera.', 'picks the bull up off the ground and tackles him and beatin the bull to the ground.', 'gets back on the horse as the bull just lays on the ground defeated.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}