{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"513e1198-03c2-5ffb-9647-7c84d4080d22","task_key":"default--validation--21878","task_revision_id":"1","upstream_id":"21878","short_description":"HellaSwag validation 21878","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a makeup artist portfolio [title] Present your makeup artist portfolio in a professional manner. [step] Purchase a professional portfolio, either purchased from an art supply store or ordered online for customization, in which you can display photos of your work. A standard black artist portfolio with your name on the front works best.\",\"endings\":\"[\\\"[title] Collect any photographs you have of your artist, including photographs taken by the artist's assistants. [title] Include a list of your work, contact information, feedback and word of mouth.\\\", '[title] Catch the attention of the client with your first photo. [step] Your first picture should be your best work and the style of make up you are most passionate about.', 'Wrap photos of your work in decorative paper to appeal to the eye, then hide your portfolio under a cover sheet to avoid exposed artwork. [substeps] If possible, limit your portfolio to three or four photographs.', \\\"[substeps] When presenting your portfolio professionally, you are replying to follow up questions regarding your work. It's up to you what can be covered and the subject matter discussed for you to have an idea of what to cover.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}