{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7d701855-c236-54e9-b290-21958c43ad76","task_key":"default--validation--20319","task_revision_id":"1","upstream_id":"20319","short_description":"HellaSwag validation 20319","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to install a door jamb [title] Measure the width of the door frame. [step] Break out the tape measure. You'll need to know how wide your door frame is so the jamb fits in it.\",\"endings\":\"[\\\"The width of the door frame will also work to determine how large your door handles should be. [substeps] Measure to within 3/4 inch (6.35 mm) of the door frame's top edge.\\\", 'Hold the tape measure up to the top of the door frame. Note the measurement and save it for later.', 'In determining the width, take the tape measure and the frame measurements and cut down to the line of lines you just cut. Then place the measuring tape so that it measures from top to bottom and across the hinge line, mixing them up to determine the width of the frame.', 'For perimeter measurements, use a 9-inch (22.9 cm) long tape measure. For interior measurements, use a 1-inch (2.5-cm) length tape measure.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}