Optimization techniques and applications with examples

• Main Author: Yang, Xin-She
• Format: eBook Book
• Language: English
• Published: Hoboken, N.J Wiley 2018; John Wiley & Sons, Incorporated; Wiley-Blackwell
• Edition: 1
• Subjects: Mathematical optimization
• ISBN: 9781119490548; 1119490545; 1119490626; 9781119490623
• DOI: 10.1002/9781119490616

A guide to modern optimization applications and techniques in newly emerging areas spanning optimization, data science, machine intelligence, engineering, and computer sciences Optimization Techniques and Applications with Examples introduces the fundamentals of all the commonly used techniquesin op...

Table of Contents:

• 3.2.3 Steepest Descent Method -- 3.2.4 Line Search -- 3.2.5 Conjugate Gradient Method -- 3.2.6 Stochastic Gradient Descent -- 3.2.7 Subgradient Method -- 3.3 Gradient-Free Nelder-Mead Method -- 3.3.1 A Simplex -- 3.3.2 Nelder-Mead Downhill Simplex Method -- Exercises -- Bibliography -- Chapter 4 Constrained Optimization -- 4.1 Mathematical Formulation -- 4.2 Lagrange Multipliers -- 4.3 Slack Variables -- 4.4 Generalized Reduced GradientMethod -- 4.5 KKT Conditions -- 4.6 Penalty Method -- Exercises -- Bibliography -- Chapter 5 Optimization Techniques: Approximation Methods -- 5.1 BFGS Method -- 5.2 Trust-Region Method -- 5.2 Trust-Region Method -- 5.3 Sequential Quadratic Programming -- 5.3.1 Quadratic Programming -- 5.3.2 SQP Procedure -- 5.4 Convex Optimization -- 5.5 Equality Constrained Optimization -- 5.6 Barrier Functions -- 5.7 Interior-Point Methods -- 5.8 Stochastic and Robust Optimization -- Exercises -- Bibliography -- Part III Applied Optimization -- Chapter 6 Linear Programming -- 6.1 Introduction -- 6.2 Simplex Method -- 6.2.1 Slack Variables -- 6.2.2 Standard Formulation -- 6.2.3 Duality -- 6.3 Worked Example by Simplex Method -- 6.4 Interior-Point Method for LP -- Exercises -- Bibliography -- Chapter 7 Integer Programming -- 7.1 Integer Linear Programming -- 7.1.1 Review of LP -- 7.1.2 Integer LP -- 7.2 LP Relaxation -- 7.3 Branch and Bound -- 7.3.1 How to Branch -- 7.4 Mixed Integer Programming -- 7.5 Applications of LP, IP, and MIP -- 7.5.1 Transport Problem -- 7.5.2 Product Portfolio -- 7.5.3 Scheduling -- 7.5.4 Knapsack Problem -- 7.5.5 Traveling Salesman Problem -- Exercises -- Bibliography -- Chapter 8 Regression and Regularization -- 8.1 Sample Mean and Variance -- 8.2 Regression Analysis -- 8.2.1 Maximum Likelihood -- 8.2.2 Regression -- 8.2.3 Linearization -- 8.2.4 Generalized Linear Regression -- 8.2.5 Goodness of Fit
• 8.3 Nonlinear Least Squares -- 8.3.1 Gauss-Newton Algorithm -- 8.3.2 Levenberg-Marquardt Algorithm -- 8.3.3 Weighted Least Squares -- 8.4 Over-fitting and Information Criteria -- 8.5 Regularization and Lasso Method -- 8.6 Logistic Regression -- 8.7 Principal Component Analysis -- Exercises -- Bibliography -- Chapter 9 Machine Learning Algorithms -- 9.1 DataMining -- 9.1.1 Hierarchy Clustering -- 9.1.2 k-Means Clustering -- 9.1.3 DistanceMetric -- 9.2 DataMining for Big Data -- 9.2.1 Characteristics of Big Data -- 9.2.2 Statistical Nature of Big Data -- 9.2.3 Mining Big Data -- 9.3 Artificial Neural Networks -- 9.3.1 Neuron Model -- 9.3.2 Neural Networks -- 9.3.3 Back Propagation Algorithm -- 9.3.4 Loss Functions in ANN -- 9.3.5 Stochastic Gradient Descent -- 9.3.6 Restricted Boltzmann Machine -- 9.4 Support Vector Machines -- 9.4.1 Statistical Learning Theory -- 9.4.2 Linear Support Vector Machine -- 9.4.3 Kernel Functions and Nonlinear SVM -- 9.5 Deep Learning -- 9.5.1 Learning -- 9.5.2 Deep Neural Nets -- 9.5.3 Tuning of Hyper-Parameters -- Exercises -- Bibliography -- Chapter 10 Queueing Theory and Simulation -- 10.1 Introduction -- 10.1.1 Components of Queueing -- 10.1.2 Notations -- 10.2 Arrival Model -- 10.2.1 Poisson Distribution -- 10.2.2 Inter-arrival Time -- 10.3 Service Model -- 10.3.1 Exponential Distribution -- 10.3.2 Service Time Model -- 10.3.3 Erlang Distribution -- 10.4 Basic Queueing Model -- 10.4.1 M/M/1 Queue -- 10.4.2 M/M/s Queue -- 10.5 Little's Law -- 10.6 Queue Management and Optimization -- Exercises -- Bibliography -- Part IV Advanced Topics -- Chapter 11 Multiobjective Optimization -- 11.1 Introduction -- 11.2 Pareto Front and Pareto Optimality -- 11.3 Choice and Challenges -- 11.4 Transformation to Single Objective Optimization -- 11.4.1 Weighted SumMethod -- 11.4.2 Utility Function -- 11.5 The -Constraint Method
• Intro -- Title Page -- Copyright Page -- Contents -- List of Figures -- List of Tables -- Preface -- Acknowledgements -- Acronyms -- Introduction -- Part I Fundamentals -- Chapter 1 Mathematical Foundations -- 1.1 Functions and Continuity -- 1.1.1 Functions -- 1.1.2 Continuity -- 1.1.3 Upper and Lower Bounds -- 1.2 Review of Calculus -- 1.2.1 Differentiation -- 1.2.2 Taylor Expansions -- 1.2.3 Partial Derivatives -- 1.2.4 Lipschitz Continuity -- 1.2.5 Integration -- 1.3 Vectors -- 1.3.1 Vector Algebra -- 1.3.2 Norms -- 1.3.3 2D Norms -- 1.4 Matrix Algebra -- 1.4.1 Matrices -- 1.4.2 Determinant -- 1.4.3 Rank of a Matrix -- 1.4.4 Frobenius Norm -- 1.5 Eigenvalues and Eigenvectors -- 1.5.1 Definiteness -- 1.5.2 Quadratic Form -- 1.6 Optimization and Optimality -- 1.6.1 Minimum and Maximum -- 1.6.2 Feasible Solution -- 1.6.3 Gradient and Hessian Matrix -- 1.6.4 Optimality Conditions -- 1.7 General Formulation of Optimization Problems -- Exercises -- Further Reading -- Chapter 2 Algorithms, Complexity, and Convexity -- 2.1 What Is an Algorithm? -- 2.2 Order Notations -- 2.3 Convergence Rate -- 2.4 Computational Complexity -- 2.4.1 Time and Space Complexity -- 2.4.2 Class P -- 2.4.3 Class NP -- 2.4.4 NP-Completeness -- 2.4.5 Complexity of Algorithms -- 2.5 Convexity -- 2.5.1 Linear and Affine Functions -- 2.5.2 Convex Functions -- 2.5.3 Subgradients -- 2.6 Stochastic Nature in Algorithms -- 2.6.1 Algorithms with Randomization -- 2.6.2 Random Variables -- 2.6.3 Poisson Distribution and Gaussian Distribution -- 2.6.4 Monte Carlo -- 2.6.5 Common Probability Distributions -- Exercises -- Bibliography -- Part II Optimization Techniques and Algorithms -- Chapter 3 Optimization -- 3.1 Unconstrained Optimization -- 3.1.1 Univariate Functions -- 3.1.2 Multivariate Functions -- 3.2 Gradient-Based Methods -- 3.2.1 Newton's Method -- 3.2.2 Convergence Analysis
• 11.6 Evolutionary Approaches -- 11.6.1 Metaheuristics -- 11.6.2 Non-Dominated Sorting Genetic Algorithm -- Exercises -- Bibliography -- Chapter 12 Constraint-Handling Techniques -- 12.1 Introduction and Overview -- 12.2 Method of Lagrange Multipliers -- 12.3 Barrier Function Method -- 12.4 Penalty Method -- 12.5 Equality Constraints via Tolerance -- 12.6 Feasibility Criteria -- 12.7 Stochastic Ranking -- 12.8 Multiobjective Constraint-Handling and Ranking -- Exercises -- Bibliography -- Part V Evolutionary Computation and Nature-Inspired Algorithms -- Chapter 13 Evolutionary Algorithms -- 13.1 Evolutionary Computation -- 13.2 Evolutionary Strategy -- 13.3 Genetic Algorithms -- 13.3.1 Basic Procedure -- 13.3.2 Choice of Parameters -- 13.4 Simulated Annealing -- 13.5 Differential Evolution -- Exercises -- Bibliography -- Chapter 14 Nature-Inspired Algorithms -- 14.1 Introduction to SI -- 14.2 Ant and Bee Algorithms -- 14.3 Particle Swarm Optimization -- 14.3.1 Accelerated PSO -- 14.3.2 Binary PSO -- 14.4 Firefly Algorithm -- 14.5 Cuckoo Search -- 14.5.1 CS Algorithm -- 14.5.2 Lévy Flight -- 14.5.3 Advantages of CS -- 14.6 Bat Algorithm -- 14.7 Flower Pollination Algorithm -- 14.8 Other Algorithms -- Exercises -- Bibliography -- Appendix A Notes on Software Packages -- A.1 Software Packages -- A.2 Matlab Codes in This Book -- A.3 Optimization Software -- A.4 DataMining Software -- A.5 Machine Learning Software -- Appendix B Problem Solutions -- Solutions for Chapter 1 -- Solutions for Chapter 2 -- Solutions for Chapter 3 -- Solutions for Chapter 4 -- Solutions for Chapter 5 -- Solutions for Chapter 6 -- Solutions for Chapter 7 -- Solutions for Chapter 8 -- Solutions for Chapter 9 -- Solutions for Chapter 10 -- Solutions for Chapter 11 -- Solutions for Chapter 12 -- Solutions for Chapter 13 -- Solutions for Chapter 14 -- Index -- EULA