{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"14ea6a4f-b827-5d96-bc02-2ae3acf35932","task_key":"default--validation--6908","task_revision_id":"1","upstream_id":"6908","short_description":"HellaSwag validation 6908","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to manage your excitement [title] Have fun to take your mind off excitement. [step] Immerse yourself in a challenging video game. Throw a ball to your dog.\",\"endings\":\"[\\\"Get on the potty or in your pajamas. If your mood isn't up, imagine your comfort level at home, and marvel at the wonders that come along with challenging play.\\\", \\\"Or, craft a simple exercise routine where you don't have time to develop your martial arts skills. [substeps] It's not what the game.\\\", 'Play peekaboo with friends, or make a comedy film. Take advantage of the great outdoors so you have something to look forward to.', 'Go on a bike ride with your younger sibling (s). Watch a whole bunch of funny youtube videos.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}