{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f309938-0f2d-59b0-82b7-c97d5fd6dffa","task_key":"default--validation--43627--variant-977bb1400cae","task_revision_id":"1","upstream_id":"43627--variant-977bb1400cae","short_description":"HellaSwag validation 43627","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to drill metal [title] Select the right bit for the job. [step] High speed steel (hss) bits will work for most kinds of metal, as will carbon steel bits coated with titanium nitride (tin). For very hard metals, use a cobalt steel bit.\",\"endings\":\"[\\\"[title] Secure a loose piece of metal you're drilling by clamping it to your workstation or setting it in a vice. [step] This step is not necessary if drilling into a large, heavy steel object like a wall or stud.\\\", 'Otherwise, use a silver or diamond bit, depending on the variety you use. Your drill bit should be inconspicuous.', 'For very soft metals, use hot steel bits. If you plan to drill hard brass brass or rubber, use mechanical steel bits.', \\\"[title] Find a good drill and read the manufacturer's instructions. [step] You will be drilling a bit like a sluice.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}