{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b63566d3-55a0-52cd-b079-e5976c1c9c07","task_key":"default--validation--40458","task_revision_id":"1","upstream_id":"40458","short_description":"HellaSwag validation 40458","config":"default","split":"validation","body":"{\"activity_label\":\"Arm wrestling\",\"ctx\":\"Two men are seen leading over the hood of the car while grabbing onto each other's hands. they\",\"endings\":\"['climb into the car while another driver is seen riding on a motorcycle and grabbing hold of people as well.', 'begin break dancing together on the sides while running in the middle with one holding the other.', 'then begin a game of arm wrestling and ends with one man winning and both looking at the camera and one holding his wrist.', 'move all along the car jumping on and off the ends while the driver drives, laughing and playing with one another.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}