{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"29a5d3ab-b144-5d45-bd33-bfe878d1a9fb","task_key":"default--validation--47191","task_revision_id":"1","upstream_id":"47191","short_description":"HellaSwag validation 47191","config":"default","split":"validation","body":"{\"activity_label\":\"Getting a piercing\",\"ctx\":\"When they are done talking, the man takes a pair of silver clamps and puts it around the girl's eyebrows. After, he inserts a needle into her left eyebrow, followed by the jewelry. once the balls\",\"endings\":\"['are screwed in, he takes a napkin and dabs away the blood before holding up a mirror for the girl to see.', 'of her eye are inserted, the man reappears to wait for the ball and add the bigger piercings.', \\\"are inserted, a piercing appears on the girl's lower lip.\\\", 'are in, they go into her forehead, and one after the other the girls begin to have seizures.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}