{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0dc7f741-8fc2-576e-9324-daf4002d69e9","task_key":"default--validation--25535","task_revision_id":"1","upstream_id":"25535","short_description":"HellaSwag validation 25535","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make dangling pearl earrings [title] Assemble the equipment listed below. [step] Make both earrings step-by-step together. Slide your beads in the following order onto both headpins: pearl, bali spacer bead, clear crystal bead, small round silver spacer bead.\",\"endings\":\"['1 bead coiled around each ear wire, then one would be placed on each ear wire. Then you can tie a knot to fix the beads in place.', 'Slide the bead into one clear glass bead holder. [title] Fit the beaded bead holder into the holder.', '[title] Use your hands and bend the headpins 80 to 90 degrees above the small round spacer beads. [title] Grip the bent headpin above the small spacer bead with your round nose pliers as shown in the graphic.', 'You need to gather enough beads for both beads to be able to fit into both headpins plus a few in each headpin. Take beads and thread each bead into your eye.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}