{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0514e962-7f20-5d44-bff0-75bd178d1a5a","task_key":"default--validation--40273","task_revision_id":"1","upstream_id":"40273","short_description":"HellaSwag validation 40273","config":"default","split":"validation","body":"{\"activity_label\":\"Playing congas\",\"ctx\":\"A four piece percussion and string band play on a stage with the main focus being a man on the bongos in the center of the stage and wearing sunshades indoors. a man\",\"endings\":\"['in a resetnases violin is then shown performing a song on the violin.', \\\"appears in the same position on stage that he was in when he's performing the set and finishes his song and walks to the end of stage and lets the show go as police officers approach to bring the event back into the field.\\\", 'carries a violin along in front of a stage backdrop for the performance.', 'plays bongos in an animated fashion while a woman next to him plays the keyboard and a man behind him plays guitar.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}