{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"89be3e25-2a28-5a6a-a271-6d9a11e79bb2","task_key":"default--validation--39693","task_revision_id":"1","upstream_id":"39693","short_description":"HellaSwag validation 39693","config":"default","split":"validation","body":"{\"activity_label\":\"Sharpening knives\",\"ctx\":\"Then he stops sharpening the knife and takes out some pieces of paper to show how sharp the knife is as he cuts slivers of paper with the knife. then another screen\",\"endings\":\"['appears with a screen with his name and picture and he explains proper tattoo technique.', \\\"appears and it's a press conference about the activities of some individuals and what type of journey they are taking now.\\\", 'appears with a computer screen and along side it encouraging words to visit two websites.', 'appears to let there not only show how fast the sharpening was achieved but also shows his single blow of the knife forward cutting part of a shirt and showing how the wooden handle of the flat knife is sharpening.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}