{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"335ac21f-3ed2-5a33-aca5-f38d3190154e","task_key":"default--validation--12477","task_revision_id":"1","upstream_id":"12477","short_description":"HellaSwag validation 12477","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to highlight hair [title] Choose the right color. [step] For highlights, you'll either need to use bleach or a dye that's lighter than your hair color, depending on your hair's color and condition. It's best to aim for a color that is one to two shades lighter than your base color.\",\"endings\":\"['Essentially, you want your hair to be slightly lighter than your base color. [substeps] It will take more than a few dye sessions to achieve your desired color.', '[substeps] The amount of highlighting you do depends on your hair type. 1.5 to 0.5 shades lighter than your base color usually goes well.', 'Going too light can create an unnatural, stripy effect. If you have the option, go for a box that comes with a toner.', 'For highlights, opt for a cool blond or caramel tint. [substeps] In addition to highlighting hair, opt for an orange or teal color, if you want as your highlights color.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}