{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"12495679-2376-539a-82bf-4535b098d555","task_key":"default--validation--24154","task_revision_id":"1","upstream_id":"24154","short_description":"HellaSwag validation 24154","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be the hottest girl at school [title] Find out what you don't like about yourself. [step] In order for other people to think you're \\\" hot, \\\" you have to believe it to be true yourself. If you don't like your skin, clear it up; if you don't like your hair, get it cut or grow it out.\",\"endings\":\"[\\\"Don't change yourself to meet someone else's standards. Instead, make yourself the kind of beautiful you want to be.\\\", \\\"[substeps] You'll probably need to deal with body odor too, and that can make you feel more unattractive. Don't drink or smoke in front of other girls.\\\", 'Say you like every other color on your body-but you\\\\'re not \\\" hot. \\\" if you like more, don\\\\'t just focus on your features.', \\\"If you don't like your nails, have them ripped. If you have blue eyes, get your nails cut short.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}