{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a4ef688a-93d0-5622-80be-f6a0ec535496","task_key":"default--validation--13470","task_revision_id":"1","upstream_id":"13470","short_description":"HellaSwag validation 13470","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do the perfect makeup for fair skin [title] Determine your skin's undertones. [substeps] You will have either warm undertones (yellow, peach or red) or cool undertones (pink, violet or blue). A quick trick is looking at the veins on the underside of your wrist.\",\"endings\":\"['They should be not there but remember that veins do not abide in all skin types-most people are fair in some areas, and the veins will stay yellow in other areas. The exception to this rule is their kidneys, which are located at the back of each eye.', 'If they are blue or purple, you have cool undertones. If they appear greenish or even yellowish, you have warm undertones.', 'If you can see veins, you have neutral undertones (or maybe both) and are probably warm-toned. Cool undertones include beige, gold, copper, with deep browns and greens.', 'This tells you how fair your skin is. Also, look at the path and shape of the veins on your wrist (in other words, what color is they? ).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}