{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"870c364a-9661-561c-b4fa-643b28f646d6","task_key":"default--validation--3335","task_revision_id":"1","upstream_id":"3335","short_description":"HellaSwag validation 3335","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make your lips stand out [title] Select three types of lipstick. [step] These should all be roughly the same color, but three different shades. Lip liner may be easier to apply for the lightest and darkest shades.\",\"endings\":\"['Avoid multi-colored lipstick or different shades of black. They will blend together a little better, but they might leave your lips looking murky and muddy.', '[substeps] The darkest shade will be used for outlining and complementing. For a bolder look, replace it with another colour, such as purple, scarlet, or vivid red.', 'Lip liner might be easier to apply for the whites. Warm tones: choose a lip liner that is slightly warmer than your lips so it will conform to your lips more fully.', 'Top lip liner may be best for more intense colors, like dark reds, blacks, greens, pinks, pinks, peaches, and purples. You can use any color or style you like.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}