{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fddcecc9-4cc1-534a-b922-a66031b4b88a","task_key":"default--validation--10277","task_revision_id":"1","upstream_id":"10277","short_description":"HellaSwag validation 10277","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to dress for spring [title] Aim for bright colors and patterns to give your outfit some style. [step] Light colors give a cheerful, happy, and fresh look to spring fashion. Dark colors, however, tend to feel like winter.\",\"endings\":\"['[substeps] Contrast the brightest colors of your outfit with lighter, more pastel garments to give your look some flair. For example, winter tends to be colder than spring, and muted colors, like cotton, mustard, and merino yellow, can lighten things up.', '[substeps] Dark color is often associated with seasons like hot and cold. Just as you might wear jeans for a formal event, dark color is also associated with seasons like election season.', '[substeps] A high crown in a warm sky can make an old hairstyle stiff. Darker shades give striking, timeless touches while lighter shades minimize them.', 'Ditch the black and navy items and go for something yellow, blue, or green instead. [substeps] Pastels are always a go to in spring fashion.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}