{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"15dd8fea-94f6-598d-8c4d-74f28cdcd8a1","task_key":"default--validation--32365","task_revision_id":"1","upstream_id":"32365","short_description":"HellaSwag validation 32365","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to look great in a yearbook [title] Keep it fresh. [step] Don't forget that your hygiene is just as important as having an amazing smile in your photo. You should take a shower and wash your face before you show those pearly whites.\",\"endings\":\"['Always keep your hair well-styled and wear it down. [substeps] If you always wear your hair down, take it down to your waist.', \\\"[substeps] If possible, don't wear makeup, especially if your your school has one. You just don't want to put on too much anyway.\\\", \\\"You should also wash your hair that is already in the photo. Don't let your hair look milky, flaky, messy, or greasy.\\\", '[substeps] If you normally shower in the evenings, try to make time for a morning shower on the day of your photo. This will make your skin look even better.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}