{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02455499-c378-5e93-8e4f-d8c49d8ec2d9","task_key":"default--validation--31817","task_revision_id":"1","upstream_id":"31817","short_description":"HellaSwag validation 31817","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose a red dress [title] Determine your skin tone. [step] Your skin tone has a big impact on what colors looks best on you, so it's important to know what type you have before choosing a shade of red. Skin tones tend to fall into three categories: warm, cool, and neutral.\",\"endings\":\"['Warm skin brings more colour into your color spectrum than cool color brings. Cool skin often has less hue and brighter colors, but cool skin tends to be prettier and have deeper shades that add color more quickly.', 'Your skin tends to fall into whichever category works for you most. [substeps] Warm skin tones are usually cool colors.', 'Warm skin tones tend to look best and cover more skin in one color. Cool skin tones are warmer colors that cover more skin, but are still more prone to darkening.', '[substeps] People with warm skin tones have yellow, peachy or golden undertones and tend to tan more easily. The veins on their inner arms appear green.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}