{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c8bcc34-4ff3-5216-9b51-231bec62b492","task_key":"default--validation--28047","task_revision_id":"1","upstream_id":"28047","short_description":"HellaSwag validation 28047","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose makeup brushes [title] Opt for brushes with natural fibers instead of synthetic fibers. [step] Organic or natural fibers are both softer and more effective. They are actual hair.\",\"endings\":\"[\\\"Even if your brushes are made of natural fibers, they're still preferable to synthetic fibers or plastics. You should have a broad range of brushes at hand.\\\", \\\"Synthetic brushes are absorbent and won't snag when pulled against your face. Because of this, you should avoid harsh, acidic bristles that could irritate your skin.\\\", \\\"Aim for natural-looking bristles, which tend to be easier to manipulate since they'll not be forced through sweat. [title] Choose brushes specifically for your hair type.\\\", \\\"They have cuticles which are better at attaching to and holding the pigment on the brush until you apply it to your face. Find cruelty free items if that's important to you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}