{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"855b13f9-a87d-5f07-8d86-2327947cb5d6","task_key":"default--validation--22488--variant-29b2dea9335f","task_revision_id":"1","upstream_id":"22488--variant-29b2dea9335f","short_description":"HellaSwag validation 22488","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to choose bohemian style home decor [title] Mix patterns. [step] One way to achieve the bohemian look in your home is to mix patterns. It's okay that it doesn't match.\",\"endings\":\"[\\\"This is because patterns don't always mesh with one another. For example, if you don't mix 3 primary colors together, then they don't mix well together.\\\", 'In fact, mixing patterns makes it easier to go back and fix them. Even minimalist minimalist furniture can still work around a bohemian theme.', 'You can go white or taupe if you want. But what color scheme does you want? Another way is to simply use different color patterns.', 'Choose patterns that you like and put them together with no care to having them match perfectly. [substeps] You may want to choose a color base, like red or blue, and only choose patterns that have that base color.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}