{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2dfb4275-e1ea-5004-82c5-7627252c1caa","task_key":"default--validation--1247","task_revision_id":"1","upstream_id":"1247","short_description":"HellaSwag validation 1247","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to sharpen hair clippers [title] Unscrew the blades. [step] Find and unscrew the screws that attach the blades to the rest of the clippers. On most hair clipper models, there are two screws at the base of the blade.\",\"endings\":\"['Once these are removed, gently detach the blades and any pieces holding them in place. [substeps] If the bottom blade does not pop out easily, use a pair of tweezers to pull it off the base.', 'Use a flathead screwdriver to loosen these screws. [substeps] The blades of the clippers will be very sharp and will pull up strands of hair from underneath your fingernails.', 'One screw is on the left and the other screw is on the right. Then slide off of the bottom of the blades.', 'This is where you will place the blades, usually on the back side of the clippers. [substeps] Note: the front of the teeth is facing in the direction you cut, not the top side of the tooth.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}