{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"004c7f45-bc42-5f07-b007-a05357668e6e","task_key":"default--validation--35362--variant-39f9809d66ef","task_revision_id":"1","upstream_id":"35362--variant-39f9809d66ef","short_description":"HellaSwag validation 35362","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply base makeup [title] Use concealer. [step] You will wreck your look if you've got dark circles under your eyes or have blemishes throughout your face. Don't feel badly about such circles or blemishes, though.\",\"endings\":\"['This is a natural part of makeup, and you can use that to your advantage. If you must draw your brows in, that will be where most of the concealer and powder will come in.', 'You can always cover them up if need be. You can also apply some false lashes-it will look better, even if you only have the eyelashes.', 'You can cover them up with concealer. Apply the concealer in an upside-down pyramid shape.', '[substeps] Use some concealer up off of your eyelids so you can wear concealer successfully without using too much. Apply concealer a little higher up on your eyelid than the same way.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}