{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"243d8d6c-69bd-552f-b447-bfdc754262cc","task_key":"default--validation--36087","task_revision_id":"1","upstream_id":"36087","short_description":"HellaSwag validation 36087","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose your best clothing colors [title] Determine your skin's undertone. [step] You will want to mostly stick to colors that match your undertone. While there are many different skin tones, there are only two undertones: warm and cool.\",\"endings\":\"['Warm skin has a yellow or orange tint, while cool skin has a blue or pale pink one. There are two main ways to figure out which category applies to you.', \\\"Warm skin tends to be brighter and brighten during the winter season, while cool colors tend to be heavily winter-heavy. Pick colors that complement your skin's undertone, rather than any other.\\\", 'The warm undertones are what give you the brightest complexion--the warm undertones mimic aging, and the cool undertones mimic aging. [substeps] If you have an undertone, experiment with different accessories.', \\\"[substeps] Be sure that your skin's undertone is within one of two categories: warm and cool. There are many different skin tones, including warm and cool.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}