{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b68b1d75-0420-5364-8ad7-a5faed99bd33","task_key":"default--validation--31849","task_revision_id":"1","upstream_id":"31849","short_description":"HellaSwag validation 31849","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to find the perfect concealer or foundation shade [title] Recognize the difference between skin undertone and skin shade. [step] Your undertone is the true color of your skin, just beneath the surface. Sun exposure, rosacea and acne can change the surface color of your skin, but the undertone never changes.\",\"endings\":\"[\\\"Your skin is different from the other person's skin, but the blood from your skin covers and lightens the true color of your skin. [title] Be aware of the variety of shades you may have.\\\", '[substeps] There are 3 possible undertones-warm, cool and neutral. Foundations and concealers are usually divided into one of those 3 categories.', \\\"By washing, avoiding bacteria, and watching your skin for change when you haven't studied at school, you can blend into your makeup seamlessly. In the beginning, you'll still be fair skinned, but with occasional adjustments, you can trim your skin until it looks and feel healthy.\\\", 'Skin undertone is known to have an undertone and will never change. You will have to experiment in order to detect whether your undertone is a color darker than your skin tone.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}