{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c9f4fc28-b5d9-5357-9898-38ffe2d9379e","task_key":"default--validation--22951","task_revision_id":"1","upstream_id":"22951","short_description":"HellaSwag validation 22951","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose a short hairstyle that suits your face shape [title] Determine your face shape. [step] The first step is to observe your facial shape intently and identify the key aspects and features that make your face stand out. To do this, stand in front of a mirror with your head tied back and an expressionless face.\",\"endings\":\"[\\\"Take note of your key features to assess your shape. Even if your face shape doesn't exactly match these guidelines, stick to what's the closest to your face shape : [substeps] An oval shaped face is slightly long and has well defined cheekbones.\\\", 'Close your eyes, and take a deep breath. Gather a snapshot of the face that is in your mind.', 'Stare at your face to judge whether it is unsymmetrical (a \\\" grin \\\"), round, rounded, or kind (with a\\\\' blank\\\\' face). [substeps] Look for your face\\\\'s key features in people with a smiley face, inverted or upside-down.', 'Try to picture yourself with your eyes slightly wider and your lips parted. [substeps] If the image appeals to you, then you have a very nice face.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}