{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"be80a7f6-5411-54c0-bb82-2425628e087c","task_key":"default--validation--22042","task_revision_id":"1","upstream_id":"22042","short_description":"HellaSwag validation 22042","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to pierce ears with a sewing needle [title] Clean the area where you'll be piercing your ears. [step] The least recommended place is the washroom as microbes from the toilet and sink faucet are contaminated. If you'd like to do your piercings there, wipe off countertops, sink faucets, toilet; in general, clean your washroom as many bacteria and viruses usually cause infection.\",\"endings\":\"['[title] Gather your materials and wash your hands well with an anti-bacterial soap and hot water. [step] Doing so will kill most bacteria on your hands.', '[title] Practice the technique with a soft, clean ear cloth. [step] The absorbent fabric is ideal for sewing your piercing while using a set size, cloth needle.', '[title] Prick the end of the piercing with a sewing needle. [step] This will help to open up the extraction area and make it easier to work the needle.', \\\"[title] Purchase a pair of sharp needle nosed pliers. [step] Your cutters will help hold up your ear area while you're waiting for your piercing to happen.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}