{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4bab9694-b56c-5ca2-92d8-7cb24ac54469","task_key":"default--validation--32129","task_revision_id":"1","upstream_id":"32129","short_description":"HellaSwag validation 32129","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to clean a scanned manga page in photoshop [title] Scan an image and open it using photoshop. [title] Rotate the image so that it is level. [substeps] Select the ruler tool: on photoshop's' tools' palette, click and hold on the eyedropper tool to access a sub-menu that includes the ruler tool.\",\"endings\":\"['The ruler tool will scan in the image and keep up with the resolution. Adjust the dimensions for each of the tab buttons.', \\\"Rotate the ruler tool until it is 110 degrees. In photoshop' tools' palette the ruler tool has a set of curves with rounded edges.\\\", 'Click on the taskbar tool: click on the toolbar tool] ] screen. If the pixels in your cursor are not aligned well with the center of the page, then delete the bunch of pixels and revisit the grid.', 'Use the ruler tool to \\\" measure \\\" a line in the scanned image that should be horizontal (or vertical). After \\\" measuring \\\" the line in step 2, choose rotate canvas and click arbitrary.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}