{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bfa7dbb4-a731-5f24-b882-de097b25f1b3","task_key":"default--validation--41192","task_revision_id":"1","upstream_id":"41192","short_description":"HellaSwag validation 41192","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to decoupage shoes [title] Choose the paper. [step] Thin to moderate weight paper works better than heavy paper, but otherwise, the only limit here is your imagination. Collect several sheets of paper in nearly any pattern or design that appeals to you.\",\"endings\":\"[\\\"[title] Trace the pattern's shape on the paper with a pen. [step] Use a series of 15-20 even, overlapping circles as the base of your pattern.\\\", 'You may think that high quality aluminum journal paper will work but the exact design may vary from one brand to another. [title] Choose the right kind of calligraphy.', '[substeps] Some good sources can include wrapping paper, old magazines, old books, comics, and crossword puzzles. If you are unable to find anything you like in print, you can also find images you like online and print them onto standard printer paper.', 'This will help you and your salespeople keep tabs on how much material is left and how many frames will do the job. [substeps] Pillowcases or plain paper work best, but patterned paper has the added benefit of protecting the shoes from paint, which will make them skinnier.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}