On the Kozinec Algorithm

• Published in: Vestnik, St. Petersburg University. Mathematics Vol. 58; no. 1; pp. 37 - 48
• Main Authors: Malozemov, V. N., Solovyeva, N. A., Tamasyan, G. Sh
• Format: Journal Article
• Language: English
• Published: Moscow Pleiades Publishing 01.03.2025; Springer Nature B.V
• Subjects: Algorithms; Analysis; Approximation; Convexity; Euclidean geometry; Hyperplanes; Machine learning; Mathematics; Mathematics and Statistics; Numerical methods; Optimality criteria; Support vector machines; Vectors (mathematics); SMO-algorithm; clipped iterations; MDM-algorithm; machine learning; Kozinec algorithm; strict SVM separation
• ISSN: 1063-4541; 1934-7855
• DOI: 10.1134/S1063454125700050

In 1964, at the onset of machine learning, Kozinec proposed a simple numerical method (algorithm) for solving the following extremum problem. “In n -dimensional Euclidean space, two finite sets P 1 and P 2 are given. It is assumed that the corresponding convex hulls C 1 and C 2 of these sets have no common points. It is required to construct a hyperplane separating the sets P 1 and P 2 , i.e., such a hyperplane that does not have common points with the sets C 1 and C 2 and, in addition, the sets C 1 and C 2 lie on opposite sides of this hyperplane. In fact, it is desirable to find, among all hyperplanes separating the sets P 1 and P 2 , such a hyperplane whose distance to the set P 1 ∪ P 2 has the maximum value. Obviously, this hyperplane will be a hyperplane passing through the middle of the vector connecting any two nearest points of the sets C 1 and C 2 , perpendicular to it.” Later this problem was called the problem of strict SVM separation of two finite sets (SVM is an abbreviation for Support Vector Machine). The Kozinec algorithm uses a natural geometric version of the optimality criterion for the problem under consideration. This paper provides a detailed analysis of the Kozinec algorithm in a modern light. In particular, a correct proof of its convergence is given. A working scheme algorithm is proposed. Two examples are considered in which the effectiveness of the conceptual and working schemes is compared.