{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a9d6f27b-1588-5201-b0a8-013538a9b103","task_key":"default--validation--48956","task_revision_id":"1","upstream_id":"48956","short_description":"HellaSwag validation 48956","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to refill a stapler [title] Open up the standard or mini stapler. [step] On most staplers, you lift the top. There may be some latches on the side that need to slide to unlock it.\",\"endings\":\"['In these cases, you\\\\'ll need to use a screwdriver. [substeps] Keeping that in mind, if you have your settings on \\\" off, \\\" there should be a picture of a click-and-release cord somewhere on the handle.', 'Consult a handbook or refer to a menu for the type of stapler you will need to use to refill your pencils. [substeps] The card stock is found beneath the teeth on most basic printer models, including those 97.3-10 and 36.5-10.', \\\"In some cases, the stapler doesn't open upward; instead, a large push button on the top rear ejects the tray out the front. [substeps] Examine the stapler you're refilling to work out which method will open the stapler tray.\\\", 'When the top opens, you will also have to unscrew some screws that are holding it in place. Some standard staplers may still not have the release key in order to unscrew these.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}