{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"615c7f34-6495-529d-877b-2205629cd6bd","task_key":"default--validation--34715","task_revision_id":"1","upstream_id":"34715","short_description":"HellaSwag validation 34715","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make paracord ranger beads [title] Fold a length of paracord into a loop. [step] You are making a shape that sort of looks like a tilted \\\" 4. \\\" the looped part should be pointing upward.\",\"endings\":\"['[title] Keep folding the \\\" 4 \\\" end until the pliers are visible at the top of the bead. [step] Keep the end where the pliers are.', 'The top edges of this loop point upwards. [title] Cut the open end of the paracord.', 'The top cord should be pointing to the right, and the bottom cord to the left. Most of the cord should be on the right side.', '[title] Start wrapping 1 end of the paracord around the pencil. [step] Wrap it around the pencil tip of the pencil \\\" 1.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}