Introduction to Optimization Methods

1 The optimization problem.- 1.1 Introduction.- 1.2 Problem definition.- 1.3 Optimization in one dimension.- 1.4 Optimization in n dimensions.- 2 Single variable optimization.- 2.1 Review of methods.- 2.2 The Fibonacci search.- 2.3 The Golden Section search.- 2.4 The Algorithm of Davies, Swann, and Campey.- 3 Multi-variable optimization.- 3.1 Introduction.- 3.2 Search methods.- 3.3 Gradient methods.- 4 Advanced methods.- 4.1 Introduction.- 4.2 General considerations.- 4.3 Advanced search methods.- 4.4 Advanced gradient methods.- 4.5 Minimax methods.- 5 Constrained optimization.- 5.1 Introduction.- 5.2 The Kuhn-Tucker conditions.- 5.3 Constrained optimization techniques.- 5.4 Direct search methods with constraints.- 5.5 Small step gradient methods.- 5.6 Sequential unconstrained methods.- 5.7 Large step gradient methods.- 5.8 Lagrangian methods.- 5.9 General considerations.- 5.10 Conclusion.- References.- Further reading.