{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"16b52b42-1592-5c84-8675-150609d32a4e","task_key":"default--validation--12448","task_revision_id":"1","upstream_id":"12448","short_description":"HellaSwag validation 12448","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to nudge in pro tools [title] Set a nudge value. [step] Setting up nudge values will allow you to determine the value of each nudge increment. [substeps] Open the nudge value menu.\",\"endings\":\"['Select nudge values from the menu that appears. Make sure that the square icon next to each nudge value stands out.', 'At the top of the screen, you will find various achievements that will give you nudge values. Each nudge trend will be identified by a column.', 'In your pro tools session, locate the nudge value menu by locating the toolbar on the right side of the main counter and below the grid value menu. Click on the downward arrow to open the nudge value menu.', 'Click the \\\" share with default \\\" or \\\" manage default \\\" options, depending on what you want to share. Click the checkboxes beneath each nudge increment type, among other things.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}