Short answers, pulled from the story.
Werner Fenchel laid the theoretical groundwork for nonlinear programming before digital computers existed. He died in 1944 and his work on convex analysis provided the essential framework for understanding how to navigate complex mathematical landscapes where straight lines and simple ratios fail to describe reality.
A problem becomes nonlinear when the objective function is not a linear function or when at least one of the constraints is not a linear equality. The mathematical definition requires that at least one of the functions f, gi, or hj be nonlinear, creating a feasible region that is not a simple polygon but a complex shape that may be disconnected or contain holes.
Every nonlinear programming problem faces one of three distinct fates: it is either feasible, infeasible, or unbounded. A feasible problem exists when there is at least one set of values for the choice variables that satisfies all constraints, while an infeasible problem occurs when the constraints are mutually contradictory, and an unbounded problem describes a feasible scenario where the objective function can be improved indefinitely.
The Karush-Kuhn-Tucker conditions serve as the necessary conditions for a solution to be optimal under differentiability and constraint qualifications. When the problem is convex, these conditions become sufficient for a global optimum, but without convexity, they guarantee only a local optimum, leaving the solver vulnerable to getting stuck in a suboptimal valley.
Numerical methods for solving nonlinear programs rely on iterative processes that start with an initial point and proceed to points that are supposed to be closer to the optimal point. These methods utilize three kinds of update rules based on the information they extract from the functions, including zero-order routines that use only values, first-order routines that incorporate gradients, and second-order routines that use Hessians to describe curvature.
Branch and bound techniques offer a powerful alternative to iterative numerical methods by dividing the program into subclasses to be solved with convex or linear approximations. This method forms a lower bound on the overall cost within the subdivision, ensuring that the true solution lies within a specific range, and the algorithm may be stopped early with the assurance that the best possible solution is within a tolerance from the best point found.