{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0da50bfe-776a-581f-a86f-8d3fabbd447e","task_key":"default--validation--35957","task_revision_id":"1","upstream_id":"35957","short_description":"HellaSwag validation 35957","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear pink eye shadow [title] Consider your skin tone. [step] Different shades of pink are more or less flattering depending on your skin tone. Before selecting a pink eye shadow, think about what shades would look best on you.\",\"endings\":\"['[substeps] If you have darker skin, a lighter shade will create a dramatic look, while a bolder shade will provide a subtle effect. For example, pale pink will stand out on your skin, but dark pinks like fuchsia or berry will look very flattering.', 'Earth tones are the most flattering for most people, yet ideal for those with sensitive skin. Pink eye shadows should go to either a brunette or redhead.', '[substeps] Subtle pink eye shadow is pricier, and works well with almost any skin tone. If you have sensitive skin, you may want to opt for a neutral shade.', 'In general, you want a pink eye shadow to be bright, neutral, and dewy. If you are happy with your skin, consider pink eye shadow to be a good color that accentuates your cheeks, elbows, and head.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}