{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d06b29fc-5167-5d9d-b177-1a26490ff8e4","task_key":"default--validation--34963","task_revision_id":"1","upstream_id":"34963","short_description":"HellaSwag validation 34963","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to arrange flowers for your wedding [title] Consider your colors. [step] If you have a specific color scheme for your wedding, pick flowers to match. Think about which types of flowers will come in the right colors for you.\",\"endings\":\"[\\\"If you're going for a warm, pastel color scheme, choose silk flowers for your wedding. If you're going for cool, tropical color schemes, stock up on cool flowers like amber or deep purple.\\\", \\\"[substeps] If you're doing something on the spectrum of blue or purple, there are many attractive colors to choose from. Things like anemones, clematises, lilacs, water lilies, tweedias, and delphiniums come in a wide range of blues and purples.\\\", '[substeps] For a more trendy look, consider decorations like centerpieces or flowers for candle holders, with plants and candles combined in the color. For a more sophisticated look, try clear paint or watercolors for centerpieces like candles, wreaths, and flowers.', '[title] Choose flowers with specific types of petals. [step] If you are having a big ceremony, the colors on petals are very important.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}