{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"99618c6f-9d7c-58cc-af2f-d20f680ba017","task_key":"default--validation--10671--variant-ee4c7b730b20","task_revision_id":"1","upstream_id":"10671--variant-ee4c7b730b20","short_description":"HellaSwag validation 10671","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to support legacy software in windows vista (program compatibility wizard ) [title] Open the feature : [substeps] Click \\\" start button. \\\" hit \\\" control panel. \\\" click \\\" programs.\",\"endings\":\"['\\\" (figure i.1) choose the \\\" use an older program with this version of windows \\\" under the \\\" programs and features \\\" heading. ( figure i.2) at this time, the \\\" start application in compatibility mode \\\" window appears with the \\\" program compatibility wizard \\\" inside [title] There are 2 options available : [substeps] Click \\\" next \\\" button to begin the wizard.', '\\\" click \\\" add/remove programs, \\\" and then \\\" windows components \\\" from the list in the \\\" options. \\\" click \\\" windows configure first \\\" and then click the \\\" advanced \\\" button.', '\\\" [step] A window will appear showing approximately the number of apps in your home screen. If there are more, navigate down and to the \\\" programs \\\" section [title] Under the \\\" background text pane \\\" slider, under the upper section you will see \\\" background text pane.', '\\\" [title] Look for the motherboard utility extension, \\\" discoverage updates will be free. [step] \\\" [title] Go through the list of installed drivers : [substeps] Click the \\\" run the every update \\\" button if any drivers are missing, and then search in \\\" all compatibility updates \\\" to see your version.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}