{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7afc8db8-6365-51ae-b40a-e8004420dcc7","task_key":"default--validation--20003","task_revision_id":"1","upstream_id":"20003","short_description":"HellaSwag validation 20003","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to divide a hexagon into three equal parts [title] Mark the center of the hexagon. [step] If the center is not marked, you can locate it using a straightedge. Draw a line (diagonal), connecting any vertex to the opposite vertex.\",\"endings\":\"['Then, take the horizontal line and trace the center across all four surfaces of the hexagon. [substeps] The outline of the ammonia and nitrate will remain in the center.', 'Note that you will be dividing one point from the vertex division for this method. [title] Make a second crease following the outer edge of the upper corner of the right corner of the right corner.', 'For example, divide 1x 250 by 1x 20. If 1x 20 equals 1x 100 x 2x 50, then the center is divided.', 'Draw a second diagonal connecting to different vertices. The intersection of these diagonals is the center point of the hexagon.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}