{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4bfcc203-f890-5c09-a38f-7123d53775e4","task_key":"default--validation--28861","task_revision_id":"1","upstream_id":"28861","short_description":"HellaSwag validation 28861","config":"default","split":"validation","body":"{\"activity_label\":\"Playing harmonica\",\"ctx\":\"As he plays, more of the young kids gather around to watch him play as he switches the two harmonicas. when he\",\"endings\":\"['is finished, the young boys begin smiling and start to clap in amazement.', 'finishes, the young kids gives each other high five.', 'is done, the dog is ready for the final dish as that is tagged with pins, and trophies.', 'is done, all of the previews are shown and they are all laughing and enjoying themselves in his bedroom.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}