{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5257906-d234-5931-acee-d1b8b65c9ae8","task_key":"default--validation--23012","task_revision_id":"1","upstream_id":"23012","short_description":"HellaSwag validation 23012","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to fax on the canon mx410 [title] Power on your canon mx410 printer. [title] Press the button labeled \\\" fax. [step] Your printer will enter fax standby mode.\",\"endings\":\"['[title] Place the documents you want faxed in the face side-up position on the platen glass located on top of your printer. [title] Press the button labeled \\\" fax quality.', '[title] Let the printer indicate the printer is fully charged. [step] This will vary but is generally preferable for whole foods or snacks.', 'On canon mx410, this mode is set to \\\" ok \\\" and will prompt the printer to enter the slack (or ip address). [title] Press the \\\" enter \\\" button to start fax standby.', 'Wait two to three seconds until the printer reboots. Click the button labeled \\\" 4fax standby....']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}