{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"abeaa746-a3a8-5101-a91f-0464be63e0ff","task_key":"default--validation--11","task_revision_id":"1","upstream_id":"11","short_description":"HellaSwag validation 11","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do monochromatic makeup [title] Choose a base color that complements your skin tone. [step] Colors that work great for the monochromatic look are red, pink, bronze, brown, nude and orange. However, if you are feeling extra adventurous, you can also choose purple, grey, blue or even green as your base color.\",\"endings\":\"[\\\"Don't worry if you are wearing different colors all the time, because wearing the same base color everyday will bring more attention to your eyes. [title] Choose a shimmery eyeshadow.\\\", '[substeps] Stick to one color or a variety of colors that pair well with your skin tone. For example, if you have fair skin, go for purple and orange.', '[substeps] For example, if you have a neutral complexion, choose salmon, yellow or light pink as your base color. If you have a yellowish complexion, go for icy yellow.', 'The colors that work best for you will depend on whether your skin has warm, cool, or neutral undertones. [substeps] If your skin has greenish, olive, or golden undertones, your skin tone is \\\" warm.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}