
          E04 -- Minimizing or Maximizing a Function    Introduction -- E04
                                    Chapter E04
                        Minimizing or Maximizing a Function

          Contents of this Introduction:

          1.     Scope of the Chapter

          2.     Background to the Problems

          2.1.   Types of Optimization Problems

          2.1.1. Unconstrained minimization

          2.1.2. Nonlinear least-squares problems

          2.1.3. Minimization subject to bounds on the variables

          2.1.4. Minimization subject to linear constraints

          2.1.5. Minimization subject to nonlinear constraints
          
          2.2.   Geometric Representation and Terminology

          2.2.1. Gradient vector

          2.2.2. Hessian matrix

          2.2.3. Jacobian matrix; matrix of constraint normals

          2.3.   Sufficient Conditions for a Solution

          2.3.1. Unconstrained minimization

          2.3.2. Minimization subject to bounds on the variables

          2.3.3. Linearly-constrained minimization

          2.3.4. Nonlinearly-constrained minimization

          2.4.   Background to Optimization Methods

          2.4.1. Methods for unconstrained optimization

          2.4.2. Methods for nonlinear least-squares problems

          2.4.3. Methods for handling constraints

          2.5.   Scaling

          2.5.1. Transformation of variables

          2.5.2. Scaling the objective function

          2.5.3. Scaling the constraints

          2.6.   Analysis of Computed Results

          2.6.1. Convergence criteria

          2.6.2. Checking results

          2.6.3. Monitoring progress

          2.6.4. Confidence intervals for least-squares solutions

          2.7.   References

          3.     Recommendations on Choice and Use of Routines

          3.1.   Choice of Routine

          3.2.   Service Routines

          3.3.   Function Evaluations at Infeasible Points

          3.4.   Related Problems



          1. Scope of the Chapter

          An optimization problem involves minimizing a function (called
          the objective function) of several variables, possibly subject to
          restrictions on the values of the variables defined by a set of
          constraint functions. The routines in the NAG Foundation Library
          are concerned with function minimization only, since the problem
          of maximizing a given function can be transformed into a
          minimization problem simply by multiplying the function by -1.

          This introduction is only a brief guide to the subject of
          optimization designed for the casual user. Anyone with a
          difficult or protracted problem to solve will find it beneficial
          to consult a more detailed text, such as Gill et al [5] or
          Fletcher [3].

          Readers who are unfamiliar with the mathematics of the subject
          may find some sections difficult at first reading; if so, they
          should concentrate on Sections 2.1, 2.2, 2.5, 2.6 and 3.

          2. Background to the Problems

          2.1. Types of Optimization Problems

          Solution of optimization problems by a single, all-purpose,
          method is cumbersome and inefficient. Optimization problems are
          therefore classified into particular categories, where each
          category is defined by the properties of the objective and
          constraint functions, as illustrated by some examples below.

          Properties of Objective   Properties of Constraints
          Function

          Nonlinear                 Nonlinear

          Sums of squares of        Sparse linear
          nonlinear functions

          Quadratic                 Linear

          Sums of squares of linear Bounds
          functions

          Linear                    None

          For instance, a specific problem category involves the
          minimization of a nonlinear objective function subject to bounds
          on the variables. In the following sections we define the
          particular categories of problems that can be solved by routines
          contained in this Chapter.

          2.1.1.  Unconstrained minimization

          In unconstrained minimization problems there are no constraints
          on the variables. The problem can be stated mathematically as
          follows:

                                    minimize F(x)
                                     x

                         n                           T
          where x is in R , that is, x=(x ,x ,...,x ) .
                                        1  2      n

          2.1.2.  Nonlinear least-squares problems

          Special consideration is given to the problem for which the
          function to be minimized can be expressed as a sum of squared
          functions. The least-squares problem can be stated mathematically
          as follows:

                                 {     m       }
                                 { T   --  2   }            n
                        minimize {f f= >  f (x)},  x is in R
                         x       {     --  i   }
                                 {     i=1     }

          where the ith element of the m-vector f is the function f (x).
                                                                   i

          2.1.3.  Minimization subject to bounds on the variables

          These problems differ from the unconstrained problem in that at
          least one of the variables is subject to a simple restriction on
          its value, e.g.x <=10, but no constraints of a more general form
                          5
          are present.

          The problem can be stated mathematically as follows:

                                                       n
                              minimize F(x),  x is in R
                               x

          subject to l <=x <=u , i=1,2,...,n.
                      i   i   i

          This format assumes that upper and lower bounds exist on all the
          variables. By conceptually allowing u =infty and l =-infty all
                                               i            i
          the variables need not be restricted.

          2.1.4.  Minimization subject to linear constraints

          A general linear constraint is defined as a constraint function
          that is linear in more than one of the variables, e.g. 3x +2x >=4
                                                                   1   2
          The various types of linear constraint are reflected in the
          following mathematical statement of the problem:


                                                       n
                              minimize F(x),  x is in R
                               x

          subject to the

                             T
             equality       a x=b        i=1,2,...,m ;
             constraints:    i   i                1

                             T
             inequality     a x>=b       i=m +1,m +2,...,m ;
             constraints:    i    i         1    1        2

                             T
                            a x<=b       i=m +1,m +2,...,m ;
                             i    i         2    2        3

                                 T
             range          s <=a x<=t   i=m +1,m +2,...,m ;
             constraints:    j   i    j     3    3        4
                                         j=1,2,...,m -m ;
                                                    4  3

             bounds         l <=x <=u    i=1,2,...,n
             constraints:    i   i   i

          where each a  is a vector of length n; b , s  and t  are constant
                      i                           i   j      j
          scalars; and any of the categories may be empty.

          Although the bounds on x  could be included in the definition of
                                  i
          general linear constraints, we prefer to distinguish between them
          for reasons of computational efficiency.

          If F(x) is a linear function, the linearly-constrained problem is
          termed a linear programming problem (LP problem); if F(x) is a
          quadratic function, the problem is termed a quadratic programming
          problem (QP problem). For further discussion of LP and QP
          problems, including the dual formulation of such problems, see
          Dantzig [2].

          2.1.5.  Minimization subject to nonlinear constraints

          A problem is included in this category if at least one constraint
                                       2
          function is nonlinear, e.g. x +x +x -2>=0. The mathematical
                                       1  3  4
          statement of the problem is identical to that for the linearly-
          constrained case, except for the addition of the following
          constraints:

             equality       c (x)=0        i=1,2,...,m ;
             constraints:    i                        5

             inequality     c (x)>=0       i=m +1,m +2,...,m ;
             constraints:    i                5    5        6

             range          v <=c (x)<=w   i=m +1,m +2,...,m ,
             constraints:    j   i      j     6    6        7
                                           j=1,2,...,m -m
                                                      7  6

          where each c  is a nonlinear function; v  and w  are constant
                      i                           j      j
          scalars; and any category may be empty. Note that we do not
          include a separate category for constraints of the form c (x)<=0,
                                                                   i
          since this is equivalent to -c (x)>=0.
                                        i

          2.2. Geometric Representation and Terminology

          To illustrate the nature of optimization problems it is useful to
          consider the following example in two dimensions

                                 x
                                  1   2   2
                           F(x)=e  (4x +2x +4x x +2x +1).
                                      1   2   1 2   2

          (This function is used as the example function in the
          documentation for the unconstrained routines.)


                                     Figure 1
                   Please see figure in printed Reference Manual

          Figure 1 is a contour diagram of F(x). The contours labelled
          F ,F ,...,F  are isovalue contours, or lines along which the
           0  1      4
                                                                   *
          function F(x) takes specific constant values. The point x  is a
                                                                *
          local unconstrained minimum, that is, the value of F(x ) is less
          than at all the neighbouring points. A function may have several
          such minima. The lowest of the local minima is termed a global
                                                            *
          minimum. In the problem illustrated in Figure 1, x  is the only
                                   

          local minimum. The point x is said to be a saddle point because
          it is a minimum along the line AB, but a maximum along CD.

          If we add the constraint x >=0 to the problem of minimizing F(x),
                                    1
          the solution remains unaltered. In Figure 1 this constraint is
          represented by the straight line passing through x =0, and the
                                                            1
          shading on the line indicates the unacceptable region. The region
              n
          in R  satisfying the constraints of an optimization problem is
          termed the feasible region. A point satisfying the constraints is
          defined as a feasible point.

          If we add the nonlinear constraint x +x -x x -1.5>=0, represented
                                              1  2  1 2
                                                       *
          by the curved shaded line in Figure 1, then x  is not a feasible
                                                                ^
          point. The solution of the new constrained problem is x, the
          feasible point with the smallest function value.

          2.2.1.  Gradient vector

          The vector of first partial derivatives of F(x) is called the
          gradient vector, and is denoted by g(x), i.e.,

                             [ ddF(x)  ddF(x)      ddF(x)]T
                        g(x)=[ ------, ------,..., ------] .
                             [  ddx     ddx         ddx  ]
                             [     1       2           n ]

          For the function illustrated in Figure 1,

                                   [      x          ]
                                   [       1         ]
                                   [F(x)+e  (8x +4x )]
                                   [           1   2 ]
                                   [ x               ]
                                   [  1              ]
                              g(x)=[e  (4x +4x +2)   ].
                                   [      2   1      ]

          The gradient vector is of importance in optimization because it
          must be zero at an unconstrained minimum of any function with
          continuous first derivatives.

          2.2.2.  Hessian matrix

          The matrix of second partial derivatives of a function is termed
          its Hessian matrix. The Hessian matrix of F(x) is denoted by G(x)
                                                  2
          and its (i,j)th element is given by dd F(x)/ddx ddx . If F(x)
                                                           i   j
          has continuous second derivatives, then G(x) must be positive
          semi-definite at any unconstrained minimum of F.

          2.2.3.  Jacobian matrix; matrix of constraint normals

          In nonlinear least-squares problems, the matrix of first partial
          derivatives of the vector-valued function f(x) is termed the
          Jacobian matrix of f(x) and its (i,j)th component is ddf /ddx .
                                                                  i    j

          The vector of first partial derivatives of the constraint c (x)
                                                                     i
          is denoted by

                                 [ ddc (x)      ddc (x)]T
                                 [    i            i   ]
                           a (x)=[ -------,..., -------] .
                            i    [  ddx          ddx   ]
                                 [     1            n  ]

                      ^                ^
          At a point, x, the vector a (x) is orthogonal (normal) to the
                                     i
                                                    ^
          isovalue contour of c (x) passing through x; this relationship is
                               i
          illustrated for a two-dimensional function in Figure 2.


                                     Figure 2
                   Please see figure in printed Reference Manual

          The matrix whose columns are the vectors {a } is termed the
                                                     i
          matrix of constraint normals. Note that if c (x) is a linear
                                                      i
                                T
          constraint involving a x, then its vector of first partial
                                i
          derivatives is simply the vector a .
                                            i

          2.3. Sufficient Conditions for a Solution

          All nonlinear functions will be assumed to have continuous second
          derivatives in the neighbourhood of the solution.

          2.3.1.  Unconstrained minimization

                                                                 *
          The following conditions are sufficient for the point x  to be an
          unconstrained local minimum of F(x):

                      *
          (i)   |||g(x )|||=0; and

                   *
          (ii)  G(x ) is positive-definite,

          where |||g||| denotes the Euclidean length of g.

          2.3.2.  Minimization subject to bounds on the variables

          At the solution of a bounds-constrained problem, variables which
          are not on their bounds are termed free variables. If it is known
          in advance which variables are on their bounds at the solution,
          the problem can be solved as an unconstrained problem in just the
          free variables; thus, the sufficient conditions for a solution
          are similar to those for the unconstrained case, applied only to
          the free variables.

                                                      *
          Sufficient conditions for a feasible point x  to be the solution
          of a bound-constrained problem are as follows:

                      *
          (i)   |||g(x )|||=0; and

                   *
          (ii)  G(x ) is positive-definite; and

                    *               *
          (iii) g (x )<0,x =u ; g (x )>0,x =l ,
                 j        j  j   j        j  j

                

          where g(x) is the gradient of F(x) with respect to the free
                         

          variables, and G(x) is the Hessian matrix of F(x) with respect to
          the free variables. The extra condition (iii) ensures that F(x)
          cannot be reduced by moving off one or more of the bounds.

          2.3.3.  Linearly-constrained minimization

          For the sake of simplicity, the following description does not
          include a specific treatment of bounds or range constraints,
          since the results for general linear inequality constraints can
          be applied directly to these cases.

                         *
          At a solution x , of a linearly-constrained problem, the
          constraints which hold as equalities are called the active or
          binding constraints. Assume that there are t active constraints
                           *          ^
          at the solution x , and let A denote the matrix whose columns are
                                                                         ^
          the columns of A corresponding to the active constraints, with b
          the vector similarly obtained from b; then

                                       ^T * ^
                                       A x =b.

          The matrix Z is defined as an n by (n-t) matrix satisfying:

                                   ^T        T
                                   A Z=0;   Z Z=I.

          The columns of Z form an orthogonal basis for the set of vectors
                                       ^
          orthogonal to the columns of A.

          Define

                      T
               g (x)=Z g(x), the projected gradient vector of F(x);
                z

                      T
               G (x)=Z G(x)Z, the projected Hessian matrix of F(x).
                z

          At the solution of a linearly-constrained problem, the projected
          gradient vector must be zero, which implies that the gradient
                    *
          vector g(x ) can be written as a linear combination of the
                                     t
                     ^           *   --          ^  ^
          columns of A, i.e., g(x )= >  (lambda) a =A(lambda). The scalar
                                     --         i i
                                     i=1
          (lambda)  is defined as the Lagrange multiplier corresponding to
                  i
          the ith active constraint. A simple interpretation of the ith
          Lagrange multiplier is that it gives the gradient of F(x) along
          the ith active constraint normal; a convenient definition of the
          Lagrange multiplier vector (although not a recommended method for
          computation) is:


                                        ^T^ -1^T   *
                              (lambda)=(A A)  A g(x ).

                                     *
          Sufficient conditions for x  to be the solution of a linearly-
          constrained problem are:

                 *                  ^T * ^
          (i)   x  is feasible, and A x =b; and

                       *                            *  ^
          (ii)  |||g (x )|||=0, or equivalently, g(x )=A(lambda); and
                    z

                    *
          (iii) G (x ) is positive-definite; and
                 z

          (iv)  (lambda) >0 if (lambda)  corresponds to a constraint
                        i              i
                ^T *  ^
                a x >=b ;
                 i     i

               (lambda) <0 if (lambda)  corresponds to a constraint
                       i              i
               ^T *  ^
               a x <=b .
                i     i

               The sign of (lambda)  is immaterial for equality
                                   i
               constraints, which by definition are always active.

          2.3.4.  Nonlinearly-constrained minimization

          For nonlinearly-constrained problems, much of the terminology is
          defined exactly as in the linearly-constrained case. The set of
          active constraints at x again means the set of constraints that
                                                                     ^
          hold as equalities at x, with corresponding definitions of c and
          ^             ^
          A: the vector c(x) contains the active constraint functions, and
                         ^
          the columns of A(x) are the gradient vectors of the active
                                                           ^
          constraints. As before, Z is defined in terms of A(x) as a matrix
          such that:


                                   ^T        T
                                   A Z=0;   Z Z=I

          where the dependence on x has been suppressed for compactness.

                                                             T
          The projected gradient vector g (x) is the vector Z g(x). At the
                                         z
                    *
          solution x  of a nonlinearly-constrained problem, the projected
          gradient must be zero, which implies the existence of Lagrange
          multipliers corresponding to the active constraints, i.e.,
             *  ^  *
          g(x )=A(x )(lambda).

          The Lagrangian function is given by:


                                                     T^
                          L(x,(lambda))=F(x)-(lambda) c(x).

          We define g (x) as the gradient of the Lagrangian function; G (x)
                     L                                                 L
                                     ^
          as its Hessian matrix, and G (x) as its projected Hessian matrix,
                                      L
                ^   T
          i.e., G =Z G Z.
                 L    L
                                     *
          Sufficient conditions for x  to be a solution of nonlinearly-
          constrained problem are:

                 *                  ^  *
          (i)   x  is feasible, and c(x )=0; and

                       *                             *  ^  *
          (ii)  |||g (x )|||=0, or, equivalently, g(x )=A(x )(lambda); and
                    z

                ^   *
          (iii) G (x ) is positive-definite; and
                 L

          (iv)  (lambda) >0 if (lambda)  corresponds to a constraint of the
                        i              i
                     ^
                form c >=0; the sign of (lambda)  is immaterial for an
                      i                         i
                equality constraint.

          Note that condition (ii) implies that the projected gradient of
                                                        *
          the Lagrangian function must also be zero at x , since the
                          T                        ^  *
          application of Z  annihilates the matrix A(x ).

          2.4. Background to Optimization Methods

          All the algorithms contained in this Chapter generate an
                               (k)                                  *
          iterative sequence {x   } that converges to the solution x  in
          the limit, except for some special problem categories (i.e.,
          linear and quadratic programming). To terminate computation of
          the sequence, a convergence test is performed to determine
          whether the current estimate of the solution is an adequate
          approximation. The convergence tests are discussed in Section 2.6

                                                     (k)
          Most of the methods construct a sequence {x   } satisfying:

                              (k+1)  (k)        (k) (k)
                             x     =x   +(alpha)   p   ,

                            (k)
          where the vector p    is termed the direction of search, and
                 (k)                                          (k)
          (alpha)    is the steplength. The steplength (alpha)    is chosen
                     (k+1)     (k)
          so that F(x     )<F(x   ).


          2.4.1.  Methods for unconstrained optimization

          The distinctions among methods arise primarily from the need to
          use varying levels of information about derivatives of F(x) in
          defining the search direction. We describe three basic approaches
          to unconstrained problems, which may be extended to other problem
          categories. Since a full description of the methods would fill
          several volumes, the discussion here can do little more than
          allude to the processes involved, and direct the reader to other
          sources for a full explanation.

          (a)   Newton-type Methods (Modified Newton Methods)

                                                              (k)
                Newton-type methods use the Hessian matrix G(x   ), or a
                                                      (k)
                finite difference approximation to G(x   ), to define the
                search direction. The routines in the Library either
                                                                      (k)
                require a subroutine that computes the elements of G(x   ),
                                       (k)
                or they approximate G(x   ) by finite differences.

                Newton-type methods are the most powerful methods available
                for general problems and will find the minimum of a
                quadratic function in one iteration. See Sections 4.4. and
                4.5.1. of Gill et al [5].

          (b)   Quasi-Newton Methods

                                                                (k)
                Quasi-Newton methods approximate the Hessian G(x   ) by a
                        (k)
                matrix B    which is modified at each iteration to include
                information obtained about the curvature of F along the
                latest search direction. Although not as robust as Newton-
                type methods, quasi-Newton methods can be more efficient
                           (k)
                because G(x   ) is not computed, or approximated by finite-
                differences. Quasi-Newton methods minimize a quadratic
                function in n iterations. See Section 4.5.2 of Gill et al
                [5].

          (c)   Conjugate-Gradient Methods

                Unlike Newton-type and quasi-Newton methods, conjugate
                gradient methods do not require the storage of an n by n
                matrix and so are ideally suited to solve large problems.
                Conjugate-gradient type methods are not usually as reliable
                or efficient as Newton-type, or quasi-Newton methods. See
                Section 4.8.3 of Gill et al [5].


          2.4.2.  Methods for nonlinear least-squares problems

          These methods are similar to those for unconstrained
          optimization, but exploit the special structure of the Hessian
          matrix to give improved computational efficiency.

          Since

                                         m
                                         --  2
                                   F(x)= >  f (x)
                                         --  i
                                         i=1

          the Hessian matrix G(x) is of the form

                                            m
                                     T      --
                          G(x)=2[J(x) J(x)+ >  f (x)G (x)],
                                            --  i    i
                                            i=1

          where J(x) is the Jacobian matrix of f(x), and G (x) is the
                                                          i
          Hessian matrix of f (x).
                             i

          In the neighbourhood of the solution, |||f(x)||| is often small
                             T
          compared to |||J(x) J(x)||| (for example, when f(x) represents
          the goodness of fit of a nonlinear model to observed data). In
                           T
          such cases, 2J(x) J(x) may be an adequate approximation to G(x),
          thereby avoiding the need to compute or approximate second
          derivatives of {f (x)}. See Section 4.7 of Gill et al [5].
                           i

          2.4.3.  Methods for handling constraints

          Bounds on the variables are dealt with by fixing some of the
          variables on their bounds and adjusting the remaining free
          variables to minimize the function. By examining estimates of the
          Lagrange multipliers it is possible to adjust the set of
          variables fixed on their bounds so that eventually the bounds
          active at the solution should be correctly identified. This type
          of method is called an active set method. One feature of such
          methods is that, given an initial feasible point, all
                          (k)
          approximations x    are feasible. This approach can be extended
          to general linear constraints. At a point, x, the set of
          constraints which hold as equalities being used to predict, or
          approximate, the set of active constraints is called the working
          set.

          Nonlinear constraints are more difficult to handle. If at all
          possible, it is usually beneficial to avoid including nonlinear
          constraints during the formulation of the problem. The methods
          currently implemented in the Library handle nonlinearly
          constrained problems either by transforming them into a sequence
          of bound constraint problems, or by transforming them into a
          sequence of quadratic programming problems. A feature of almost
                                                         (k)
          all methods for nonlinear constraints is that x    is not
          guaranteed to be feasible except in the limit, and this is
          certainly true of the routines currently in the Library. See
          Chapter 6, particularly Section 6.4 and Section 6.5 of Gill et al
          [5].

          Anyone interested in a detailed description of methods for
          optimization should consult the references.

          2.5. Scaling

          Scaling (in a broadly defined sense) often has a significant
          influence on the performance of optimization methods. Since
          convergence tolerances and other criteria are necessarily based
          on an implicit definition of 'small' and 'large', problems with
          unusual or unbalanced scaling may cause difficulties for some
          algorithms. Nonetheless, there are currently no scaling routines
          in the Library, although the position is under constant review.
          In light of the present state of the art, it is considered that
          sensible scaling by the user is likely to be more effective than
          any automatic routine. The following sections present some
          general comments on problem scaling.

          2.5.1.  Transformation of variables

          One method of scaling is to transform the variables from their
          original representation, which may reflect the physical nature of
          the problem, to variables that have certain desirable properties
          in terms of optimization. It is generally helpful for the
          following conditions to be satisfied:

          (i)   the variables are all of similar magnitude in the region of
                interest;

          (ii)  a fixed change in any of the variables results in similar
                changes in F(x). Ideally, a unit change in any variable
                produces a unit change in F(x);

          (iii) the variables are transformed so as to avoid cancellation
                error in the evaluation of F(x).

          Normally, users should restrict themselves to linear
          transformations of variables, although occasionally nonlinear
          transformations are possible. The most common such transformation
          (and often the most appropriate) is of the form

                                     x   =Dx   ,
                                      new   old

          where D is a diagonal matrix with constant coefficients. Our
          experience suggests that more use should be made of the
          transformation

                                    x   =Dx   +v,
                                     new   old

          where v is a constant vector.

          Consider, for example, a problem in which the variable x
                                                                  3
          represents the position of the peak of a Gaussian curve to be
          fitted to data for which the extreme values are 150 and 170;
          therefore x  is known to lie in the range 150--170. One possible
                     3
                                                    

          scaling would be to define a new variable x , given by
                                                     3


                                          x
                                           3
                                      x = ---.
                                       3  170

                                                                 

          A better transformation, however, is given by defining x  as
                                                                  3

                                         x -160
                                          3
                                     x = ------.
                                      3    10

          Frequently, an improvement in the accuracy of evaluation of F(x)
          can result if the variables are scaled before the routines to
          evaluate F(x) are coded. For instance, in the above problem just
          mentioned of Gaussian curve fitting, x  may always occur in terms
                                                3
          of the form (x -x ), where x  is a constant representing the mean
                        3  m          m
          peak position.

          2.5.2.  Scaling the objective function

          The objective function has already been mentioned in the
          discussion of scaling the variables. The solution of a given
          problem is unaltered if F(x) is multiplied by a positive
          constant, or if a constant value is added to F(x). It is
          generally preferable for the objective function to be of the
          order of unity in the region of interest; thus, if in the
                                                                +5
          original formulation F(x) is always of the order of 10   (say),
                                                           -5
          then the value of F(x) should be multiplied by 10   when
          evaluating the function within the optimization routines. If a
          constant is added or subtracted in the computation of F(x),
          usually it should be omitted - i.e., it is better to formulate
                   2  2                 2  2               2  2
          F(x) as x +x  rather than as x +x +1000 or even x +x +1. The
                   1  2                 1  2               1  2
          inclusion of such a constant in the calculation of F(x) can
          result in a loss of significant figures.

          2.5.3.  Scaling the constraints

          The solution of a nonlinearly-constrained problem is unaltered if
          the ith constraint is multiplied by a positive weight w . At the
                                                                 i
          approximation of the solution determined by a Library routine,
          the active constraints will not be satisfied exactly, but will
                                                 -8       -6
          have 'small' values (for example, c =10  , c =10  , etc.). In
                                             1        2
          general, this discrepancy will be minimized if the constraints
          are weighted so that a unit change in x produces a similar change
          in each constraint.

          A second reason for introducing weights is related to the effect
          of the size of the constraints on the Lagrange multiplier
          estimates and, consequently, on the active set strategy.
          Additional discussion is given in Gill et al [5].

          2.6. Analysis of Computed Results

          2.6.1.  Convergence criteria

          The convergence criteria inevitably vary from routine to routine,
          since in some cases more information is available to be checked
          (for example, is the Hessian matrix positive-definite?), and
          different checks need to be made for different problem categories
          (for example, in constrained minimization it is necessary to
          verify whether a trial solution is feasible). Nonetheless, the
          underlying principles of the various criteria are the same; in
          non-mathematical terms, they are:

                                  (k)
          (i)   is the sequence {x   } converging?

                                  (k)
          (ii)  is the sequence {F   } converging?

          (iii) are the necessary and sufficient conditions for the
                solution satisfied?

          The decision as to whether a sequence is converging is
          necessarily speculative. The criterion used in the present
          routines is to assume convergence if the relative change
          occurring between two successive iterations is less than some
          prescribed quantity. Criterion (iii) is the most reliable but
          often the conditions cannot be checked fully because not all the
          required information may be available.

          2.6.2.  Checking results

          Little a priori guidance can be given as to the quality of the
          solution found by a nonlinear optimization algorithm, since no
          guarantees can be given that the methods will always work.
          Therefore, it is necessary for the user to check the computed
          solution even if the routine reports success. Frequently a '
          solution' may have been found even when the routine does not
          report a success. The reason for this apparent contradiction is
          that the routine needs to assess the accuracy of the solution.
          This assessment is not an exact process and consequently may be
          unduly pessimistic. Any 'solution' is in general only an
          approximation to the exact solution, and it is possible that the
          accuracy specified by the user is too stringent.

          Further confirmation can be sought by trying to check whether or
          not convergence tests are almost satisfied, or whether or not
          some of the sufficient conditions are nearly satisfied. When it
          is thought that a routine has returned a non-zero value of IFAIL
          only because the requirements for 'success' were too stringent it
          may be worth restarting with increased convergence tolerances.

          For nonlinearly-constrained problems, check whether the solution
          returned is feasible, or nearly feasible; if not, the solution
          returned is not an adequate solution.

          Confidence in a solution may be increased by resolving the
          problem with a different initial approximation to the solution.
          See Section 8.3 of Gill et al [5] for further information.

          2.6.3.  Monitoring progress

          Many of the routines in the Chapter have facilities to allow the
          user to monitor the progress of the minimization process, and
          users are encouraged to make use of these facilities. Monitoring
          information can be a great aid in assessing whether or not a
          satisfactory solution has been obtained, and in indicating
          difficulties in the minimization problem or in the routine's
          ability to cope with the problem.

          The behaviour of the function, the estimated solution and first
          derivatives can help in deciding whether a solution is acceptable
          and what to do in the event of a return with a non-zero value of
          IFAIL.

          2.6.4.  Confidence intervals for least-squares solutions

          When estimates of the parameters in a nonlinear least-squares
          problem have been found, it may be necessary to estimate the
          variances of the parameters and the fitted function. These can be
          calculated from the Hessian of F(x) at the solution.

          In many least-squares problems, the Hessian is adequately
                                              T
          approximated at the solution by G=2J J (see Section 2.4.3). The
          Jacobian, J, or a factorization of J is returned by all the
          comprehensive least-squares routines and, in addition, a routine
          is supplied in the Library to estimate variances of the
          parameters following the use of most of the nonlinear least-
                                                 T
          squares routines, in the case that G=2J J is an adequate
          approximation.

          Let H be the inverse of G, and S be the sum of squares, both
                                     

          calculated at the solution x; an unbiased estimate of the
          variance of the ith parameter x  is
                                         i


                                           2S
                                   var x = ---H
                                        i  m-n ii

                                                               

          and an unbiased estimate of the covariance of x  and x  is
                                                         i      j

                                              2S
                                covar(x ,x )= ---H  .
                                       i  j   m-n ij

              *
          If x  is the true solution, then the 100(1-(beta)) confidence
                      

          interval on x is

                 

                /                          * 
          x -  / var x .t                <x <x
           i \/       i  (1-(beta)/2,m-n)  i  i
               

              /    
          +  / var x .t                ,   i=1,2,...,n
           \/       i  (1-(beta)/2,m-n)

          where t                 is the 100(1-(beta))/2 percentage point
                 (1-(beta)/2,m-n)
          of the t-distribution with m-n degrees of freedom.

          In the majority of problems, the residuals f , for i=1,2,...,m,
                                                      i
          contain the difference between the values of a model function
          (phi)(z,x) calculated for m different values of the independent
          variable z, and the corresponding observed values at these
          points. The minimization process determines the parameters, or
                                                                         

          constants x, of the fitted function (phi)(z,x). For any value, z,
          of the independent variable z, an unbiased estimate of the
          variance of (phi) is


                                 n   n
                             2S  --  -- [ dd(phi)] [ dd(phi)]
                  var (phi)= --- >   >  [ -------] [ -------] H  .
                             m-n --  -- [  ddx   ] [  ddx   ]  ij
                                 i=1 j=1[     i  ]z[     j  ]z

                                                                  

          The 100(1-(beta)) confidence interval on F at the point z is


                                                          *
          (phi)(z,x)-\/var (phi).t               < (phi)(z,x )
                                  ((beta)/2,m-n)
                                             

                             < (phi)(z,x) +\/var (phi).t              .
                                                       ((beta)/2,m-n)

          For further details on the analysis of least-squares solutions
          see Bard [1] and Wolberg [7].

          2.7. References

          [1]   Bard Y (1974) Nonlinear Parameter Estimation. Academic
                Press.

          [2]   Dantzig G B (1963) Linear Programming and Extensions.
                Princeton University Press.

          [3]   Fletcher R (1987) Practical Methods of Optimization. Wiley
                (2nd Edition).

          [4]   Gill P E and Murray W (eds) (1974) Numerical Methods for
                Constrained Optimization. Academic Press.

          [5]   Gill P E, Murray W and Wright M H (1981) Practical
                Optimization. Academic Press.

          [6]   Murray W (ed) (1972) Numerical Methods for Unconstrained
                Optimization. Academic Press.

          [7]   Wolberg J R (1967) Prediction Analysis. Van Nostrand.

          3. Recommendations on Choice and Use of Routines

          The choice of routine depends on several factors: the type of
          problem (unconstrained, etc.); the level of derivative
          information available (function values only, etc.); the
          experience of the user (there are easy-to-use versions of some
          routines); whether or not storage is a problem; and whether
          computational time has a high priority.

          3.1. Choice of Routine

          Routines are provided to solve the following types of problem:


          Nonlinear Programming                                      E04UCF
          Quadratic Programming                                      E04NAF
          Linear Programming                                         E04MBF
          Nonlinear Function                                         E04DGF
          (using 1st derivatives)
          Nonlinear Function, unconstrained or simple bounds         E04JAF
          (using function values only)
          Nonlinear least-squares                                    E04FDF
          (using function values only)
          Nonlinear least-squares                                    E04GCF
          (using function values and 1st derivatives)

          E04UCF can be used to solve unconstrained, bound-constrained and
          linearly-constrained problems.

          E04NAF can be used as a comprehensive linear programming solver;
          however, in most cases the easy-to-use routine E04MBFwill be
          adequate.

          E04MBF can be used to obtain a feasible point for a set of linear
          constraints.

          E04DGF can be used to solve large scale unconstrained problems.

          The routines can be used to solve problems in a single variable.

          3.2. Service Routines

          One of the most common errors in use of optimization routines is
          that the user's subroutines incorrectly evaluate the relevant
          partial derivatives. Because exact gradient information normally
          enhances efficiency in all areas of optimization, the user should
          be encouraged to provide analytical derivatives whenever
          possible. However, mistakes in the computation of derivatives can
          result in serious and obscure run-time errors, as well as
          complaints that the Library routines are incorrect.

          E04UCF incorporates a check on the gradients being supplied and
          users are encouraged to utilize this option; E04GCF also
          incorporates a call to a derivative checker.

          E04YCF estimates selected elements of the variance-covariance
          matrix for the computed regression parameters following the use
          of a nonlinear least-squares routine.

          3.3. Function Evaluations at Infeasible Points

          Users must not assume that the routines for constrained problems
          will require the objective function to be evaluated only at
          points which satisfy the constraints, i.e., feasible points. In
          the first place some of the easy-to-use routines call a service
          routine which will evaluate the objective function at the user-
          supplied initial point, and at neighbouring points (to check
          user-supplied derivatives or to estimate intervals for finite
          differencing). Apart from this, all routines will ensure that any
          evaluations of the objective function occur at points which
          approximately satisfy any simple bounds or linear constraints.
          Satisfaction of such constraints is only approximate because:

          (a)   routines which have a parameter FEATOL may allow such
                constraints to be violated by a margin specified by FEATOL;

          (b)   routines which estimate derivatives by finite differences
                may require function evaluations at points which just
                violate such constraints even though the current iteration
                just satisfies them.

          There is no attempt to ensure that the current iteration
          satisfies any nonlinear constraints. Users who wish to prevent
          their objective function being evaluated outside some known
          region (where it may be undefined or not practically computable),
          may try to confine the iteration within this region by imposing
          suitable simple bounds or linear constraints (but beware as this
          may create new local minima where these constraints are active).

          Note also that some routines allow the user-supplied routine to
          return a parameter (MODE) with a negative value to force an
          immediate clean exit from the minimization when the objective
          function cannot be evaluated.

          3.4. Related Problems

          Apart from the standard types of optimization problem, there are
          other related problems which can be solved by routines in this or
          other chapters of the Library.

          E04MBF can be used to find a feasible point for a set of linear
          constraints and simple bounds.

          Two routines in Chapter F04 solve linear least-squares problems,
                          m                         n
                          --      2                 --
          i.e., minimize  >  r (x)  where r (x)=b - >  a  x .
                          --  i            i     i  --  ij j
                          i=1                       j=1

          E02GAF solves an overdetermined system of linear equations in the
                                    m
                                    --
          l  norm, i.e., minimizes  >  |r (x)|, with r  as above.
           1                        --   i            i
                                    i=1


          E04 -- Minimizing or Maximizing a Function        Contents -- E04
          Chapter E04

          Minimizing or Maximizing a Function

          E04DGF  Unconstrained minimum, pre-conditioned conjugate gradient
                  algorithm, function of several variables using 1st
                  derivatives

          E04DJF  Read optional parameter values for E04DGF from external
                  file

          E04DKF  Supply optional parameter values to E04DGF

          E04FDF  Unconstrained minimum of a sum of squares, combined
                  Gauss-Newton and modified Newton algorithm using function
                  values only

          E04GCF  Unconstrained minimum of a sum of squares, combined
                  Gauss-Newton and quasi-Newton algorithm, using 1st
                  derivatives

          E04JAF  Minimum, function of several variables, quasi-Newton
                  algorithm, simple bounds, using function values only

          E04MBF  Linear programming problem

          E04NAF  Quadratic programming problem

          E04UCF  Minimum, function of several variables, sequential QP
                  method, nonlinear constraints, using function values and
                  optionally 1st derivatives

          E04UDF  Read optional parameter values for E04UCF from external
                  file

          E04UEF  Supply optional parameter values to E04UCF

          E04YCF  Covariance matrix for nonlinear least-squares problem

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
   
        E04 -- Minimizing or Maximizing a Function                 E04DGF
                E04DGF -- NAG Foundation Library Routine Document
   
        Note: Before using this routine, please read the Users' Note for
        your implementation to check implementation-dependent details.
        The symbol (*) after a NAG routine name denotes a routine that is
        not included in the Foundation Library.
   
        Note for users via the AXIOM system: the interface to this routine
        has been enhanced for use with AXIOM and is slightly different to
        that offered in the standard version of the Foundation Library.  In
        particular, the optional parameters of the NAG routine are now
        included in the parameter list.  These are described in section
        5.1.2, below.
   
        1. Purpose
   
        E04DGF minimizes an unconstrained nonlinear function of several
        variables using a pre-conditioned, limited memory quasi-Newton
        conjugate gradient method. First derivatives are required. The
        routine is intended for use on large scale problems.
   
        2. Specification
   
               SUBROUTINE E04DGF(N,OBJFUN,ITER,OBJF,OBJGRD,X,IWORK,WORK,IUSER,
              1                  USER,ES,FU,IT,LIN,LIST,MA,OP,PR,STA,STO,
              2                  VE,IFAIL)
               INTEGER           N, ITER, IWORK(N+1), IUSER(*),
              1                  IT, PR, STA, STO, VE, IFAIL
               DOUBLE PRECISION OBJF, OBJGRD(N), X(N), WORK(13*N), USER(*)
              1                  ES, FU, LIN, OP, MA
               LOGICAL           LIST
               EXTERNAL          OBJFUN
   
        3. Description
   
        E04DGF uses a pre-conditioned conjugate gradient method and is
        based upon algorithm PLMA as described in Gill and Murray [1] and
        Gill et al [2] Section 4.8.3.
   
        The algorithm proceeds as follows:
   
        Let x  be a given starting point and let k denote the current
             0
        iteration, starting with k=0. The iteration requires g , the
                                                              k
        gradient vector evaluated at x , the kth estimate of the minimum.
                                      k
        At each iteration a vector p  (known as the direction of search)
                                    k
        is computed and the new estimate x    is given by x +(alpha) p
                                          k+1              k        k k
        where (alpha)  (the step length) minimizes the function
                     k
        F(x +(alpha) p ) with respect to the scalar (alpha) . A choice of
           k        k k                                    k
        initial step (alpha)  is taken as
                            0
   
                                                    T
                         (alpha) =min{1,2|F -F   |/g g }
                                0          k  est   k k
   
        where F    is a user-supplied estimate of the function value at
               est
        the solution. If F    is not specified, the software always
                          est
        chooses the unit step length for (alpha) . Subsequent step length
                                                0
        estimates are computed using cubic interpolation with safeguards.
   
        A quasi-Newton method can be used to compute the search direction
        p  by updating the inverse of the approximate Hessian (H ) and
         k                                                      k
        computing
   
                               p   =-H   g                            (1)
                                k+1   k+1 k+1
   
        The updating formula for the approximate inverse is given by
   
                                               (    T    )
                                               (   y H y )
                       1  (     T    T  )   1  (    k k k)   T
             H   =H - ----(H y s +s y H )+ ----(1+ ------)s s         (2)
              k+1  k   T  ( k k k  k k k)   T  (     T   ) k k
                      y s                  y s (    y s  )
                       k k                  k k(     k k )
   
        where y =g   -g  and s =x   -x =(alpha) p .
               k  k-1  k      k  k+1  k        k k
   
        The method used by E04DGF to obtain the search direction is based
        upon computing p    as -H   g    where H    is a matrix obtained
                        k+1      k+1 k+1        k+1
        by updating the identity matrix with a limited number of quasi-
        Newton corrections. The storage of an n by n matrix is avoided by
        storing only the vectors that define the rank two corrections -
        hence the term limited-memory quasi-Newton method. The precise
        method depends upon the number of updating vectors stored. For
        example, the direction obtained with the 'one-step' limited
        memory update is given by (1) using (2) with H  equal to the
                                                      k
        identity matrix, viz.
                                                   T    (    T  )
                                                  s g   (   y y )
                          1  ( T        T      )   k k+1(    k k)
             p   =-g   + ----(s g   y +y g   s )- ------(1+ ----)s
              k+1   k+1   T  ( k k+1 k  k k+1 k)    T   (    T  ) k
                         y s                       y s  (   y s )
                          k k                       k k (    k k)
   
        E04DGF uses a two-step method described in detail in Gill and
        Murray [1] in which restarts and pre-conditioning are
        incorporated. Using a limited-memory quasi-Newton formula, such
        as the one above, guarantees p    to be a descent direction if
                                      k+1
                                T
        all the inner products y  are positive for all vectors y  and s
                                k                               k      k
        used in the updating formula.
   
        The termination criterion of E04DGF is as follows:
   
        Let (tau)  specify a parameter that indicates the number of
                 F
        correct figures desired in F  ((tau)  is equivalent to Optimality
                                    k       F
        Tolerance in the optional parameter list, see Section 5.1). If
        the following three conditions are satisfied
   
             (i) F   -F <(tau) (1+|F |)
                  k-1  k      F     k
   
                                 ______
             (ii) ||x   -x ||<  /(tau)  (1+||x ||)
                     k-1  k   \/      F       k
   
                               ______
             (iii) ||g ||<= 3 /(tau)  (1+|F |) or ||g ||<(epsilon) ,
                      k     \/      F      k         k            A
             where (epsilon)  is the absolute error associated with
                            A
             computing the objective function
   
        then the algorithm is considered to have converged. For a full
        discussion on termination criteria see Gill et al [2] Chapter 8.
   
        4. References
   
        [1]   Gill P E and Murray W (1979) Conjugate-gradient Methods for
              Large-scale Nonlinear Optimization. Technical Report SOL 79-
              15. Department of Operations Research, Stanford University.
   
        [2]   Gill P E, Murray W and Wright M H (1981) Practical
              Optimization. Academic Press.
   
        5. Parameters
   
         1:  N -- INTEGER                                           Input
             On entry: the number n of variables. Constraint: N >= 1.
   
         2:  OBJFUN -- SUBROUTINE, supplied by the user.
                                                       External Procedure
             OBJFUN must calculate the objective function F(x) and its
             gradient for a specified n element vector x.
   
             Its specification is:
   
                    SUBROUTINE OBJFUN (MODE, N, X, OBJF, OBJGRD,
                   1                   NSTATE, IUSER, USER)
                    INTEGER          MODE, N, NSTATE, IUSER(*)
                    DOUBLE PRECISION X(N), OBJF, OBJGRD(N), USER(*)
   
              1:  MODE -- INTEGER                            Input/Output
                  MODE is a flag that the user may set within OBJFUN to
                  indicate a failure in the evaluation of the objective
                  function. On entry: MODE is always non-negative. On
                  exit: if MODE is negative the execution of E04DGF is
                  terminated with IFAIL set to MODE.
   
              2:  N -- INTEGER                                      Input
                  On entry: the number n of variables.
   
              3:  X(N) -- DOUBLE PRECISION array                    Input
                  On entry: the point x at which the objective function
                  is required.
   
              4:  OBJF -- DOUBLE PRECISION                         Output
                  On exit: the value of the objective function F at the
                  current point x.
   
              5:  OBJGRD(N) -- DOUBLE PRECISION array              Output
                                                                ddF
                  On exit: OBJGRD(i) must contain the value of  ---- at
                                                                ddx
                                                                   i
                  the point x, for i=1,2,...,n.
   
              6:  NSTATE -- INTEGER                                 Input
                  On entry: NSTATE will be 1 on the first call of OBJFUN
                  by E04DGF, and is 0 for all subsequent calls. Thus, if
                  the user wishes, NSTATE may be tested within OBJFUN in
                  order to perform certain calculations once only. For
                  example the user may read data or initialise COMMON
                  blocks when NSTATE = 1.
   
              7:  IUSER(*) -- INTEGER array                User Workspace
   
              8:  USER(*) -- DOUBLE PRECISION array        User Workspace
                  OBJFUN is called from E04DGF with the parameters IUSER
                  and USER as supplied to E04DGF. The user is free to use
                  arrays IUSER and USER to supply information to OBJFUN
                  as an alternative to using COMMON.
             OBJFUN must be declared as EXTERNAL in the (sub)program
             from which E04DGF is called. Parameters denoted as
             Input must not be changed by this procedure.
   
         3:  ITER -- INTEGER                                       Output
             On exit: the number of iterations performed.
   
         4:  OBJF -- DOUBLE PRECISION                              Output
             On exit: the value of the objective function F(x) at the
             final iterate.
   
         5:  OBJGRD(N) -- DOUBLE PRECISION array                   Output
             On exit: the objective gradient at the final iterate.
   
         6:  X(N) -- DOUBLE PRECISION array                  Input/Output
             On entry: an initial estimate of the solution. On exit: the
             final estimate of the solution.
   
         7:  IWORK(N+1) -- INTEGER array                        Workspace
   
         8:  WORK(13*N) -- DOUBLE PRECISION array               Workspace
   
         9:  IUSER(*) -- INTEGER array                     User Workspace
             Note: the dimension of the array IUSER must be at least 1.
             This array is not used by E04DGF, but is passed directly to
             routine OBJFUN and may be used to supply information to
             OBJFUN.
   
        10:  USER(*) -- DOUBLE PRECISION array             User Workspace
             Note: the dimension of the array USER must be at least 1.
             This array is not used by E04DGF, but is passed directly to
             routine OBJFUN and may be used to supply information to
             OBJFUN.
   
        11:  IFAIL -- INTEGER                                Input/Output
             On entry: IFAIL must be set to 0, -1 or 1. Users who are
             unfamiliar with this parameter should refer to the Essential
             Introduction for details.
   
             On exit: IFAIL = 0 unless the routine detects an error or
             gives a warning (see Section 6).
   
             For this routine, because the values of output parameters
             may be useful even if IFAIL /=0 on exit, users are
             recommended to set IFAIL to -1 before entry. It is then
             essential to test the value of IFAIL on exit.
   
   
        5.1. Optional Input Parameters
   
        Several optional parameters in E04DGF define choices in the
        behaviour of the routine. In order to reduce the number of formal
        parameters of E04DGF these optional parameters have associated
        default values (see Section 5.1.3) that are appropriate for most
        problems. Therefore the user need only specify those optional
        parameters whose values are to be different from their default
        values.
   
        The remainder of this section can be skipped by users who wish to
        use the default values for all optional parameters. A complete
        list of optional parameters and their default values is given in
        Section 5.1.3.
   
        5.1.1. Specification of the Optional Parameters
   
        Optional parameters may be specified by calling one, or both, of
        E04DJF and E04DKF prior to a call to E04DGF.
   
        E04DJF reads options from an external options file, with Begin
        and End as the first and last lines respectively and each
        intermediate line defining a single optional parameter. For
        example,
   
              Begin
                Print Level = 1
              End
   
        The call
   
              CALL E04DJF(IOPTNS, INFORM)
   
        can then be used to read the file on unit IOPTNS. INFORM will be
        zero on successful exit. E04DJF should be consulted for a full
        description of this method of supplying optional parameters.
   
        E04DKF can be called to supply options directly, one call being
        necessary for each optional parameter.
   
        For example,
   
   
              CALL E04DKF(`Print level = 1')
   
   
        E04DKF should be consulted for a full description of this method
        of supplying optional parameters.
   
        All optional parameters not specified by the user are set to
        their default values. Optional parameters specified by the user
        are unaltered by E04DGF (unless they define invalid values) and
        so remain in effect for subsequent calls to E04DGF, unless
        altered by the user.
   
        5.1.2. Description of the Optional Parameters
   
        The following list (in alphabetical order) gives the valid
        options. For each option, we give the keyword, any essential
        optional qualifiers, the default value, and the definition. The
        minimum valid abbreviation of each keyword is underlined. If no
        characters of an optional qualifier are underlined, the qualifier
        may be omitted. The letter a denotes a phrase (character string)
        that qualifies an option. The letters i and r denote INTEGER and
        real values required with certain options. The number (epsilon)
        is a generic notation for machine precision, and (epsilon)
                                                                  R
        denotes the relative precision of the objective function (the
        optional parameter Function Precision; see below).
   
        Defaults
   
        This special keyword may be used to reset the default values
        following a call to E04DGF.
   
        Estimated Optimal Function Value r
   
           (Axiom parameter ES)
   
        This value of r specifies the user-supplied guess of the optimum
        objective function value. This value is used by E04DGF to
        calculate an initial step length (see Section 3). If the value of
        r is not specified by the user (the default), then this has the
        effect of setting the initial step length to unity. It should be
        noted that for badly scaled functions a unit step along the
        steepest descent direction will often compute the function at
        very large values of x.
   
                                                 0.9
        Function Precision  r Default = (epsilon)
   
           (Axiom parameter FU)
   
        The parameter defines (epsilon) , which is intended to be a
                                       R
        measure of the accuracy with which the problem function F can be
        computed. The value of (epsilon)  should reflect the relative
                                        R
        precision of 1+|F(x)|; i.e. (epsilon)  acts as a relative
                                             R
        precision when |F| is large, and as an absolute precision when
        |F| is small. For example, if F(x) is typically of order 1000 and
        the first six significant digits are known to be correct, an
        appropriate value for (epsilon)  would be 1.0E-6. In contrast, if
                                       R
                                     -4
        F(x) is typically of order 10   and the first six significant
        digits are known to be correct, an appropriate value for
        (epsilon)  would be 1.0E-10. The choice of (epsilon)  can be
                 R                                          R
        quite complicated for badly scaled problems; see Chapter 8 of
        Gill and Murray [2], for a discussion of scaling techniques. The
        default value is appropriate for most simple functions that are
        computed with full accuracy. However when the accuracy of the
        computed function values is known to be significantly worse than
        full precision, the value of (epsilon)  should be large enough so
                                              R
        that E04DGF will not attempt to distinguish between function
        values that differ by less than the error inherent in the
        calculation. If 0<=r<(epsilon), where (epsilon) is the machine
        precision then the default value is used.
   
        Iteration Limit  i Default = max(50,5n)
   
        Iters
   
        Itns
   
           (Axiom parameter IT)
   
        The value i (i>=0) specifies the maximum number of iterations
        allowed before termination. If i<0 the default value is used. See
        Section 8 for further information.
   
        Linesearch Tolerance  r Default = 0.9
   
           (Axiom parameter LIN)
   
        The value r (0<=r<1) controls the accuracy with which the step
        (alpha) taken during each iteration approximates a minimum of the
        function along the search direction (the smaller the value of r,
        the more accurate the linesearch). The default value r=0.9
        requests an inaccurate search, and is appropriate for most
        problems. A more accurate search may be appropriate when it is
        desirable to reduce the number of iterations - for example, if
        the objective function is cheap to evaluate.
   
        List     Default =  List
        Nolist
   
           (Axiom parameter LIST)
   
        Normally each optional parameter specification is printed as it
        is supplied. Nolist may be used to suppress the printing and List
        may be used to restore printing.
                                           10
        Maximum Step Length  r Default = 10
   
           (Axiom parameter MA)
   
        The value r (r>0) defines the maximum allowable step length for
        the line search. If r<=0 the default value is used.
   
                                                   0.8
        Optimality Tolerance  r Default = (epsilon)
   
           (Axiom parameter OP)
                                                   R
   
        The parameter r ((epsilon) <=r<1) specifies the accuracy to which
                                  R
        the user wishes the final iterate to approximate a solution of
        the problem. Broadly speaking, r indicates the number of correct
        figures desired in the objective function at the solution. For
                           - 6
        example, if r is 10    and E04DGF terminates successfully, the
        final value of F should have approximately six correct figures.
        E04DGF will terminate successfully if the iterative sequence of x
        -values is judged to have converged and the final point satisfies
        the termination criteria (see Section 3, where (tau)  represents
                                                            F
        Optimality Tolerance).
   
        Print Level  i Default = 10
   
           (Axiom parameter PR)
   
        The value i controls the amount of printout produced by E04DGF.
        The following levels of printing are available.
   
        i     Output.
   
        0     No output.
   
        1     The final solution.
   
        5     One line of output for each iteration.
   
        10    The final solution and one line of output for each
              iteration.
   
        Start Objective Check at Variable  i Default = 1
   
           (Axiom parameter STA)
   
        Stop Objective Check at Variable  i Default = n
   
           (Axiom parameter STO)
   
        These keywords take effect only if Verify Level > 0 (see below).
        They may be used to control the verification of gradient elements
        computed by subroutine OBJFUN. For example if the first 30
        variables appear linearly in the objective, so that the
        corresponding gradient elements are constant, then it is
        reasonable to specify Start Objective Check at Variable  31.
   
        Verify Level  i Default = 0
   
        Verify                           No
   
        Verify Level                     -1
   
        Verify Level                     0
   
        Verify
   
        Verify                           Yes
   
        Verify Objective Gradients
   
        Verify Gradients
   
        Verify Level                     1
   
           (Axiom parameter VE)
   
        These keywords refer to finite-difference checks on the gradient
        elements computed by the user-provided subroutine OBJFUN. It is
        possible to set Verify Level in several ways, as indicated above.
        For example, the gradients will be verified if Verify, Verify
        Yes, Verify Gradients, Verify Objective Gradients or Verify Level
        = 1 is specified.
   
        If i<0 then no checking will be performed. If i>0 then the
        gradients will be verified at the user-supplied point. If i=0
        only a 'cheap' test will be performed, requiring one call to
        OBJFUN. If i=1, a more reliable (but more expensive) check will
        be made on individual gradient components, within the ranges
        specified by the Start and Stop keywords as described above. A
        result of the form OK or BAD? is printed by E04DGF to indicate
        whether or not each component appears to be correct.
   
        5.1.3. Optional parameter checklist and default values
   
        For easy reference, the following sample list shows all valid
        keywords and their default values. The default options Function
        Precision and Optimality Tolerance depend upon (epsilon), the
        machine precision.
   
        Optional Parameters          Default Values
   
        Estimated Optimal Function
        Value
   
                                              0.9
        Function precision           (epsilon)
   
        Iterations                   max(50,5n)
   
        Linesearch Tolerance         0.9
   
                                       10
        Maximum Step Length          10
   
        List/Nolist                  List
   
                                              0.8
        Optimality Tolerance         (epsilon)
   
        Print Level                  10
   
        Start Objective Check at     1
        Variable
   
        Stop Objective Check at      n
        Variable
   
        Verify Level                 0
   
        5.2. Description of Printed Output
   
        The level of printed output from E04DGF is controlled by the user
        (see the description of Print Level in Section 5.1).
   
        When Print Level >= 5, the following line of output is produced
        at each iteration.
   
        Itn            is the iteration count.
   
        Step           is the step (alpha) taken along the computed
                       search direction. On reasonably well-behaved
                       problems, the unit step will be taken as the
                       solution is approached.
   
        Nfun           is the cumulated number of evaluations of the
                       objective function needed for the linesearch.
                       Evaluations needed for the estimation of the
                       gradients by finite differences are not included.
                       Nfun is printed as a guide to the amount of work
                       required for the linesearch. E04DGF will perform
                       at most 16 function evaluations per iteration.
   
        Objective      is the value of the objective function.
   
        Norm G         is the Euclidean norm of the gradient of the
                       objective function.
   
        Norm X         is the Euclidean norm of x.
   
        Norm (X(k-1)-X(k)) is the Euclidean norm of x   -x .
                                                     k-1  k
   
        When Print Level = 1 or Print Level >= 10 then the solution at
        the end of execution of E04DGF is printed out.
   
        The following describes the printout for each variable:
   
        Variable       gives the name (VARBL) and index j (j = 1 to n) of
                       the variable
   
        Value          is the value of the variable at the final iterate
   
        Gradient Value is the value of the gradient of the objective
                       function with respect to the jth variable at the
                       final iterate
   
        6. Error Indicators and Warnings
   
        Errors or warnings specified by the routine:
   
        If on entry IFAIL = 0 or -1, explanatory error messages are
        output on the current error message unit (as defined by X04AAF).
   
        On exit from E04DGF, IFAIL should be tested. If Print Level > 0
        then a short description of IFAIL is printed.
   
        Errors and diagnostics indicated by IFAIL from E04DGF are as
        follows:
   
        IFAIL< 0
             A negative value of IFAIL indicates an exit from E04DGF
             because the user set MODE negative in routine OBJFUN. The
             value of IFAIL will be the same as the user's setting of
             MODE.
   
        IFAIL= 1
             Not used by this routine.
   
        IFAIL= 2
             Not used by this routine.
   
        IFAIL= 3
             The maximum number of iterations has been performed. If the
             algorithm appears to be making progress the iterations value
             may be too small (see Section 5.1.2) so the user should
             increase iterations and rerun E04DGF. If the algorithm seems
             to be 'bogged down',the user should check for incorrect
             gradients or ill-conditioning as described below under IFAIL
             = 6.
   
        IFAIL= 4
             The computed upper bound on the step length taken during the
             linesearch was too small. A rerun with an increased value of
             the Maximum Step Length ((rho) say) may be successful unless
                      10
             (rho)>=10   (the default value), in which case the current
             point cannot be improved upon.
   
        IFAIL= 5
             Not used by this routine.
   
        IFAIL= 6
             A sufficient decrease in the function value could not be
             attained during the final linesearch. If the subroutine
             OBJFUN computes the function and gradients correctly, then
             this may occur because an overly stringent accuracy has been
             requested, i.e., Optimality Tolerance is too small or if the
             minimum lies close to a step length of zero. In this case
             the user should apply the four tests described in Section 3
             to determine whether or not the final solution is acceptable
             (the user will need to set Print Level >= 5). For a
             discussion of attainable accuracy see Gill and Murray [2].
   
             If many iterations have occurred in which essentially no
             progress has been made or E04DGF has failed to move from the
             initial point, subroutine OBJFUN may be incorrect. The user
             should refer to the comments below under IFAIL = 7 and check
             the gradients using the Verify parameter. Unfortunately,
             there may be small errors in the objective gradients that
             cannot be detected by the verification process. Finite-
             difference approximations to first derivatives are
             catastrophically affected by even small inaccuracies.
   
        IFAIL= 7
             Large errors were found in the derivatives of the objective
             function. This value of IFAIL will occur if the verification
             process indicated that at least one gradient component had
             no correct figures. The user should refer to the printed
             output to determine which elements are suspected to be in
             error.
   
             As a first step, the user should check that the code for the
             objective values is correct - for example, by computing the
             function at a point where the correct value is known.
             However, care should be taken that the chosen point fully
             tests the evaluation of the function. It is remarkable how
             often the values x=0 or x=1 are used to test function
             evaluation procedures, and how often the special properties
             of these numbers make the test meaningless.
   
             Special care should be used in this test if computation of
             the objective function involves subsidiary data communicated
             in COMMON storage. Although the first evaluation of the
             function may be correct, subsequent calculations may be in
             error because some of the subsidiary data has accidentally
             been overwritten.
   
             Errors in programming the function may be quite subtle in
             that the function value is 'almost' correct. For example,
             the function may not be accurate to full precision because
             of the inaccurate calculation of a subsidiary quantity, or
             the limited accuracy of data upon which the function
             depends. A common error on machines where numerical
             calculations are usually performed in double precision is to
             include even one single-precision constant in the
             calculation of the function; since some compilers do not
             convert such constants to double precision, half the correct
             figures may be lost by such a seemingly trivial error.
   
        IFAIL= 8
             The gradient (g) at the starting point is too small. The
                    T
             value g g is less than (epsilon) |F(x )|, where (epsilon)
                                             m    o                   m
             is the machine precision.
   
             The problem should be rerun at a different starting point.
   
        IFAIL= 9
             On entry N < 1.
   
        7. Accuracy
   
        On successful exit the accuracy of the solution will be as
        defined by the optional parameter Optimality Tolerance.
   
        8. Further Comments
   
        Problems whose Hessian matrices at the solution contain sets of
        clustered eigenvalues are likely to be minimized in significantly
        fewer than n iterations. Problems without this property may
        require anything between n and 5n iterations, with approximately
        2n iterations being a common figure for moderately difficult
        problems.
   
        9. Example
   
        To find a minimum of the function
   
                              x
                               1   2   2
                           F=e  (4x +2x +4x x +2x +1).
                                   1   2   1 2   2
   
        The example program is not reproduced here. The source code for
        all example programs is distributed with the NAG Foundation
        Library software and should be available on-line.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

          E04 -- Minimizing or Maximizing a Function                 E04DJF
                  E04DJF -- NAG Foundation Library Routine Document

          Note: Before using this routine, please read the Users' Note for
          your implementation to check implementation-dependent details.
          The symbol (*) after a NAG routine name denotes a routine that is
          not included in the Foundation Library.

          1. Purpose

          To supply optional parameters to E04DGF from an external file.

          2. Specification

                 SUBROUTINE E04DJF (IOPTNS, INFORM)
                 INTEGER          IOPTNS, INFORM

          3. Description

          E04DJF may be used to supply values for optional parameters to
          E04DGF. E04DJF reads an external file and each line of the file
          defines a single optional parameter. It is only necessary to
          supply values for those parameters whose values are to be
          different from their default values.

          Each optional parameter is defined by a single character string
          of up to 72 characters, consisting of one or more items. The
          items associated with a given option must be separated by spaces,
          or equal signs (=). Alphabetic characters may be upper or lower
          case. The string

                Print level = 1

          is an example of a string used to set an optional parameter. For
          each option the string contains one or more of the following
          items:

          (a)   A mandatory keyword.

          (b)   A phrase that qualifies the keyword.

          (c)   A number that specifies an INTEGER or real value. Such
                numbers may be up to 16 contiguous characters in Fortran
                77's I, F, E or D formats, terminated by a space if this is
                not the last item on the line.

          Blank strings and comments are ignored. A comment begins with an
          asterisk (*) and all subsequent characters in the string are
          regarded as part of the comment.

          The file containing the options must start with begin and must
          finish with end An example of a valid options file is:

                Begin * Example options file
                  Print level = 10
                End


          Normally each line of the file is printed as it is read, on the
          current advisory message unit (see X04ABF), but printing may be
          suppressed using the keyword nolist. To suppress printing of
          begin, nolist must be the first option supplied as in the file:

                Begin
                  Nolist
                  Print level = 10
                End

          Printing will automatically be turned on again after a call to
          E04DGF and may be turned on again at any time by the user by
          using the keyword list.

          Optional parameter settings are preserved following a call to
          E04DGF, and so the keyword defaults is provided to allow the user
          to reset all the optional parameters to their default values
          prior to a subsequent call to E04DGF.

          A complete list of optional parameters, their abbreviations,
          synonyms and default values is given in Section 5.1 of the
          routine document for E04DGF.

          4. References

          None.

          5. Parameters

           1:  IOPTNS -- INTEGER                                      Input
               On entry: IOPTNS must be the unit number of the options
               file. Constraint: 0 <= IOPTNS <= 99.

           2:  INFORM -- INTEGER                                     Output
               On exit: INFORM will be zero if an options file with the
               correct structure has been read. Otherwise INFORM will be
               positive. Positive values of INFORM indicate that an options
               file may not have been successfully read as follows:
               INFORM = 1
                     IOPTNS is not in the range [0,99].

               INFORM = 2
                     begin was found, but end-of-file was found before end
                     was found.

               INFORM = 3
                     end-of-file was found before begin was found.

          6. Error Indicators and Warnings

          If a line is not recognised as a valid option, then a warning
          message is output on the current advisory message unit (see
          X04ABF).

          7. Accuracy

          Not applicable.

          8. Further Comments

          E04DKF may also be used to supply optional parameters to E04DGF.

          9. Example

          See the example for E04DGF.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

          E04 -- Minimizing or Maximizing a Function                 E04DKF
                  E04DKF -- NAG Foundation Library Routine Document

          Note: Before using this routine, please read the Users' Note for
          your implementation to check implementation-dependent details.
          The symbol (*) after a NAG routine name denotes a routine that is
          not included in the Foundation Library.

          1. Purpose

          To supply individual optional parameters to E04DGF.

          2. Specification

                 SUBROUTINE E04DKF (STRING)
                 CHARACTER*(*)    STRING

          3. Description

          E04DKF may be used to supply values for optional parameters to
          E04DGF. It is only necessary to call E04DKF for those parameters
          whose values are to be different from their default values. One
          call to E04DKF sets one parameter value.

          Each optional parameter is defined by a single character string
          of up to 72 characters, consisting of one or more items. The
          items associated with a given option must be separated by spaces,
          or equal signs (=). Alphabetic characters may be upper or lower
          case. The string

                Print Level = 1

          is an example of a string used to set an optional parameter. For
          each option the string contains one or more of the following
          items:

          (a)   A mandatory keyword.

          (b)   A phrase that qualifies the keyword.

          (c)   A number that specifies an INTEGER or real value. Such
                numbers may be up to 16 contiguous characters in Fortran
                77's I, F, E or D formats, terminated by a space if this is
                not the last item on the line.

          Blank strings and comments are ignored. A comment begins with an
          asterisk (*) and all subsequent characters in the string are
          regarded as part of the comment.

          Normally, each user-specified option is printed as it is defined,
          on the current advisory message unit (see X04ABF), but this
          printing may be suppressed using the keyword nolist Thus the
          statement

               CALL E04DKF (`Nolist')

          suppresses printing of this and subsequent options. Printing will
          automatically be turned on again after a call to E04DGF, and may
          be turned on again at any time by the user, by using the keyword
          list.

          Optional parameter settings are preserved following a call to
          E04DGF, and so the keyword defaults is provided to allow the user
          to reset all the optional parameters to their default values by
          the statement,

               CALL E04DKF (`Defaults')

          prior to a subsequent call to E04DGF.

          A complete list of optional parameters, their abbreviations,
          synonyms and default values is given in Section 5.1 of the
          routine document for E04DGF.

          4. References

          None.

          5. Parameters

           1:  STRING -- CHARACTER*(*)                                Input
               On entry: STRING must be a single valid option string. See
               Section 3 above, and Section 5.1 of the routine document for
               E04DGF.

          6. Error Indicators and Warnings

          If the parameter STRING is not recognised as a valid option
          string, then a warning message is output on the current advisory
          message unit (see X04ABF).

          7. Accuracy

          Not applicable.

          8. Further Comments

          E04DJF may also be used to supply optional parameters to E04DGF.

          9. Example

          See the example for E04DGF.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

          E04 -- Minimizing or Maximizing a Function                 E04FDF
                  E04FDF -- NAG Foundation Library Routine Document

          Note: Before using this routine, please read the Users' Note for
          your implementation to check implementation-dependent details.
          The symbol (*) after a NAG routine name denotes a routine that is
          not included in the Foundation Library.

          1. Purpose

          E04FDF is an easy-to-use algorithm for finding an unconstrained
          minimum of a sum of squares of m nonlinear functions in n
          variables (m>=n). No derivatives are required.

          It is intended for functions which are continuous and which have
          continuous first and second derivatives (although it will usually
          work even if the derivatives have occasional discontinuities).

          2. Specification

                 SUBROUTINE E04FDF (M, N, X, FSUMSQ, IW, LIW, W, LW, IFAIL)
                 INTEGER          M, N, IW(LIW), LIW, LW, IFAIL
                 DOUBLE PRECISION X(N), FSUMSQ, W(LW)

          3. Description

          This routine is essentially identical to the subroutine LSNDN1 in
          the National Physical Laboratory Algorithms Library. It is
          applicable to problems of the form

                                            m
                                            --        2
                             Minimize F(x)= >  [f (x)]
                                            --   i
                                            i=1

                                T
          where x=(x ,x ,...,x )  and m>=n. (The functions f (x) are often
                    1  2      n                             i
          referred to as 'residuals'.) The user must supply a subroutine
          LSFUN1 to evaluate functions f (x) at any point x.
                                        i

          From a starting point supplied by the user, a sequence of points
          is generated which is intended to converge to a local minimum of
          the sum of squares. These points are generated using estimates of
          the curvature of F(x).

          4. References

          [1]   Gill P E and Murray W (1978) Algorithms for the Solution of
                the Nonlinear Least-squares Problem. SIAM J. Numer. Anal. 15
                977--992.

          5. Parameters

           1:  M -- INTEGER                                           Input

           2:  N -- INTEGER                                           Input
               On entry: the number m of residuals f (x), and the number n
                                                    i
               of variables, x . Constraint: 1 <= N <= M.
                              j

           3:  X(N) -- DOUBLE PRECISION array                  Input/Output
               On entry: X(j) must be set to a guess at the jth component
               of the position of the minimum, for j=1,2,...,n. On exit:
               the lowest point found during the calculations. Thus, if
               IFAIL = 0 on exit, X(j) is the jth component of the position
               of the minimum.

           4:  FSUMSQ -- DOUBLE PRECISION                            Output
               On exit: the value of the sum of squares, F(x),
               corresponding to the final point stored in X.

           5:  IW(LIW) -- INTEGER array                           Workspace

           6:  LIW -- INTEGER                                         Input
               On entry: the length of IW as declared in the (sub)program
               from which E04FDF has been called. Constraint: LIW >= 1.

           7:  W(LW) -- DOUBLE PRECISION array                    Workspace

           8:  LW -- INTEGER                                          Input
               On entry: the length of W as declared in the (sub)program
               from which E04FDF is called. Constraints:
                    LW >= N*(7 + N + 2*M + (N-1)/2) + 3*M, if N > 1,

                    LW >= 9 + 5*>M, if N = 1.

           9:  IFAIL -- INTEGER                                Input/Output
               On entry: IFAIL must be set to 0, -1 or 1. Users who are
               unfamiliar with this parameter should refer to the Essential
               Introduction for details.

               On exit: IFAIL = 0 unless the routine detects an error or
               gives a warning (see Section 6).

               For this routine, because the values of output parameters
               may be useful even if IFAIL /=0 on exit, users are
               recommended to set IFAIL to -1 before entry. It is then
               essential to test the value of IFAIL on exit.

          5.1. Optional Parameters

               LSFUN1 -- SUBROUTINE, supplied by the user.
                                                         External Procedure
               This routine must be supplied by the user to calculate the
               vector of values f (x) at any point x. Since the routine is
                                 i
               not a parameter to E04FDF, it must be called LSFUN1. It
               should be tested separately before being used in conjunction
               with E04FDF (see the Chapter Introduction).

               Its specification is:

                      SUBROUTINE LSFUN1 (M, N, XC, FVECC)
                      INTEGER          M, N
                      DOUBLE PRECISION XC(N), FVECC(M)

                1:  M -- INTEGER                                      Input

                2:  N -- INTEGER                                      Input
                    On entry: the numbers m and n of residuals and
                    variables, respectively.

                3:  XC(N) -- DOUBLE PRECISION array                   Input
                    On entry: the point x at which the values of the f
                                                                       i
                    are required.

                4:  FVECC(M) -- DOUBLE PRECISION array               Output
                    On exit: FVECC(i) must contain the value of f  at the
                                                                 i
                    point x, for i=1,2,...,m.
               LSFUN1 must be declared as EXTERNAL in the (sub)program
               from which E04FDF is called. Parameters denoted as
               Input must not be changed by this procedure.

          6. Error Indicators and Warnings

          Errors or warnings specified by the routine:

          If on entry IFAIL = 0 or -1, explanatory error messages are
          output on the current error message unit (as defined by X04AAF).

          IFAIL= 1
               On entry N < 1,

               or       M < N,

               or       LIW < 1,

               or       LW < N*(7 + N + 2*M + (N-1)/2) + 3*M, when N > 1,

               or       LW < 9 + 5*>M, when N = 1.

          IFAIL= 2
               There have been 400*n calls of LSFUN1, yet the algorithm
               does not seem to have converged. This may be due to an
               awkward function or to a poor starting point, so it is worth
               restarting E04FDF from the final point held in X.

          IFAIL= 3
               The final point does not satisfy the conditions for
               acceptance as a minimum, but no lower point could be found.

          IFAIL= 4
               An auxiliary routine has been unable to complete a singular
               value decomposition in a reasonable number of sub-
               iterations.

          IFAIL= 5

          IFAIL= 6

          IFAIL= 7

          IFAIL= 8
               There is some doubt about whether the point x found by
               E04FDF is a minimum of F(x). The degree of confidence in the
               result decreases as IFAIL increases. Thus when IFAIL = 5, it
               is probable that the final x gives a good estimate of the
               position of a minimum, but when IFAIL = 8 it is very
               unlikely that the routine has found a minimum.

          If the user is not satisfied with the result (e.g. because IFAIL
          lies between 3 and 8), it is worth restarting the calculations
          from a different starting point (not the point at which the
          failure occurred) in order to avoid the region which caused the
          failure. Repeated failure may indicate some defect in the
          formulation of the problem.

          7. Accuracy

          If the problem is reasonably well scaled and a successful exit is
          made, then, for a computer with a mantissa of t decimals, one
          would expect to get about t/2-1 decimals accuracy in the
          components of x and between t-1 (if F(x) is of order 1 at the
          minimum) and 2t-2 (if F(x) is close to zero at the minimum)
          decimals accuracy in F(x).

          8. Further Comments

          The number of iterations required depends on the number of
          variables, the number of residuals and their behaviour, and the
          distance of the starting point from the solution. The number of
          multiplications performed per iteration of E04FDF varies, but for
                                   2    3
          m>>n is approximately n*m +O(n ). In addition, each iteration
          makes at least n+1 calls of LSFUN1. So, unless the residuals can
          be evaluated very quickly, the run time will be dominated by the
          time spent in LSFUN1.

          Ideally, the problem should be scaled so that the minimum value
          of the sum of squares is in the range (0,1), and so that at
          points a unit distance away from the solution the sum of squares
          is approximately a unit value greater than at the minimum. It is
          unlikely that the user will be able to follow these
          recommendations very closely, but it is worth trying (by
          guesswork), as sensible scaling will reduce the difficulty of the
          minimization problem, so that E04FDF will take less computer
          time.

          When the sum of squares represents the goodness of fit of a
          nonlinear model to observed data, elements of the variance-
          covariance matrix of the estimated regression coefficients can be
          computed by a subsequent call to E04YCF, using information
          returned in segments of the workspace array W. See E04YCF for
          further details.

          9. Example

          To find least-squares estimates of x , x  and x  in the model
                                              1   2      3

                                            t
                                             1
                                   y=x + ---------
                                      1  x t +x t
                                          2 2  3 3

          using the 15 sets of data given in the following table.

                                  y    t   t    t
                                        1   2    3
                                 0.14  1.0 15.0 1.0
                                 0.18  2.0 14.0 2.0
                                 0.22  3.0 13.0 3.0
                                 0.25  4.0 12.0 4.0
                                 0.29  5.0 11.0 5.0
                                 0.32  6.0 10.0 6.0
                                 0.35  7.0  9.0 7.0
                                 0.39  8.0  8.0 8.0
                                 0.37  9.0  7.0 7.0
                                 0.58 10.0  6.0 6.0
                                 0.73 11.0  5.0 5.0
                                 0.96 12.0  4.0 4.0
                                 1.34 13.0  3.0 3.0
                                 2.10 14.0  2.0 2.0
                                 4.39 15.0  1.0 1.0

          The program uses (0.5, 1.0, 1.5) as the initial guess at the
          position of the minimum.

          The example program is not reproduced here. The source code for
          all example programs is distributed with the NAG Foundation
          Library software and should be available on-line.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

          E04 -- Minimizing or Maximizing a Function                 E04GCF
                  E04GCF -- NAG Foundation Library Routine Document

          Note: Before using this routine, please read the Users' Note for
          your implementation to check implementation-dependent details.
          The symbol (*) after a NAG routine name denotes a routine that is
          not included in the Foundation Library.

          1. Purpose

          E04GCF is an easy-to-use quasi-Newton algorithm for finding an
          unconstrained minimum of a sum of squares of m nonlinear
          functions in n variables (m>=n). First derivatives are required.

          It is intended for functions which are continuous and which have
          continuous first and second derivatives (although it will usually
          work even if the derivatives have occasional discontinuities).

          2. Specification

                 SUBROUTINE E04GCF (M, N, X, FSUMSQ, IW, LIW, W, LW, IFAIL)
                 INTEGER          M, N, IW(LIW), LIW, LW, IFAIL
                 DOUBLE PRECISION X(N), FSUMSQ, W(LW)

          3. Description

          This routine is essentially identical to the subroutine LSFDQ2 in
          the National Physical Laboratory Algorithms Library. It is
          applicable to problems of the form

                                             m
                                             --        2
                             Minimize  F(x)= >  [f (x)]
                                             --   i
                                             i=1

                                T
          where x=(x ,x ,...,x )  and m>=n. (The functions f (x) are often
                    1  2      n                             i
          referred to as 'residuals'.) The user must supply a subroutine
          LSFUN2 to evaluate the residuals and their first derivatives at
          any point x.

          Before attempting to minimize the sum of squares, the algorithm
          checks LSFUN2 for consistency. Then, from a starting point
          supplied by the user, a sequence of points is generated which is
          intended to converge to a local minimum of the sum of squares.
          These points are generated using estimates of the curvature of
          F(x).

          4. References

          [1]   Gill P E and Murray W (1978) Algorithms for the Solution of
                the Nonlinear Least-squares Problem. SIAM J. Numer. Anal. 15
                977--992.

          5. Parameters

           1:  M -- INTEGER                                           Input

           2:  N -- INTEGER                                           Input
               On entry: the number m of residuals f (x), and the number n
                                                    i
               of variables, x . Constraint: 1 <= N <= M.
                              j

           3:  X(N) -- DOUBLE PRECISION array                  Input/Output
               On entry: X(j) must be set to a guess at the jth component
               of the position of the minimum, for j=1,2,...,n. The routine
               checks the first derivatives calculated by LSFUN2 at the
               starting point, and so is more likely to detect an error in
               the user's routine if the initial X(j) are non-zero and
               mutually distinct. On exit: the lowest point found during
               the calculations. Thus, if IFAIL = 0 on exit, X(j) is the j
               th component of the position of the minimum.

           4:  FSUMSQ -- DOUBLE PRECISION                            Output
               On exit: the value of the sum of squares, F(x),
               corresponding to the final point stored in X.

           5:  IW(LIW) -- INTEGER array                           Workspace

           6:  LIW -- INTEGER                                         Input
               On entry: the length of IW as declared in the (sub)program
               from which E04GCF is called. Constraint: LIW >= 1.

           7:  W(LW) -- DOUBLE PRECISION array                    Workspace

           8:  LW -- INTEGER                                          Input
               On entry: the length of W as declared in the (sub)program
               from which E04GCF is called. Constraints:
                    LW >= 2*N*(4 + N + M) + 3*M, if N > 1,

                    LW >= 11 + 5*M, if N = 1.

           9:  IFAIL -- INTEGER                                Input/Output
               On entry: IFAIL must be set to 0, -1 or 1. Users who are
               unfamiliar with this parameter should refer to the Essential
               Introduction for details.

               On exit: IFAIL = 0 unless the routine detects an error or
               gives a warning (see Section 6).

               For this routine, because the values of output parameters
               may be useful even if IFAIL /=0 on exit, users are
               recommended to set IFAIL to -1 before entry. It is then
               essential to test the value of IFAIL on exit.

          5.1. Optional Parameters

               LSFUN2 -- SUBROUTINE, supplied by the user.
                                                         External Procedure
               This routine must be supplied by the user to calculate the
               vector of values f (x) and the Jacobian matrix of first
                                 i
                            ddf
                               i
               derivatives  ---- at any point x. Since the routine is not a
                            ddx
                               j
               parameter to E04GCF, it must be called LSFUN2. It should be
               tested separately before being used in conjunction with
               E04GCF (see the Chapter Introduction).

               Its specification is:

                      SUBROUTINE LSFUN2 (M, N, XC, FVECC, FJACC, LJC)
                      INTEGER          M, N, LJC
                      DOUBLE PRECISION XC(N), FVECC(M), FJACC(LJC,N)
               Important: The dimension declaration for FJACC must
               contain the variable LJC, not an integer constant.

                1:  M -- INTEGER                                      Input

                2:  N -- INTEGER                                      Input
                    On entry: the numbers m and n of residuals and
                    variables, respectively.

                3:  XC(N) -- DOUBLE PRECISION array                   Input
                    On entry: the point x at which the values of the f
                                                                       i
                             ddf
                                i
                    and the  ---- are required.
                             ddx
                                j

                4:  FVECC(M) -- DOUBLE PRECISION array               Output
                    On exit: FVECC(i) must contain the value of f  at the
                                                                  i
                    point x, for i=1,2,...,m.

                5:  FJACC(LJC,N) -- DOUBLE PRECISION array           Output
                                                                    ddf
                                                                       i
                    On exit: FJACC(i,j) must contain the value of  ---- at
                                                                    ddx
                                                                       j
                    the point x, for i=1,2,...,m; j=1,2,...,n.

                6:  LJC -- INTEGER                                    Input
                    On entry: the first dimension of the array FJACC.
               LSFUN2 must be declared as EXTERNAL in the (sub)program
               from which E04GCF is called. Parameters denoted as
               Input must not be changed by this procedure.

          6. Error Indicators and Warnings

          Errors or warnings specified by the routine:

          If on entry IFAIL = 0 or -1, explanatory error messages are
          output on the current error message unit (as defined by X04AAF).

          IFAIL= 1
               On entry N < 1,

               or       M < N,

               or       LIW < 1,

               or       LW < 2*N*(4 + N + M) + 3*M, when N > 1,

               or       LW < 9 + 5*>M, when N = 1.

          IFAIL= 2
               There have been 50*n calls of LSFUN2, yet the algorithm does
               not seem to have converged. This may be due to an awkward
               function or to a poor starting point, so it is worth
               restarting E04GCF from the final point held in X.

          IFAIL= 3
               The final point does not satisfy the conditions for
               acceptance as a minimum, but no lower point could be found.

          IFAIL= 4
               An auxiliary routine has been unable to complete a singular
               value decomposition in a reasonable number of sub-
               iterations.

          IFAIL= 5

          IFAIL= 6

          IFAIL= 7

          IFAIL= 8
               There is some doubt about whether the point X found by
               E04GCF is a minimum of F(x). The degree of confidence in the
               result decreases as IFAIL increases. Thus, when IFAIL = 5,
               it is probable that the final x gives a good estimate of the
               position of a minimum, but when IFAIL = 8 it is very
               unlikely that the routine has found a minimum.

          IFAIL= 9
               It is very likely that the user has made an error in forming
                                ddf
                                   i
               the derivatives  ---- in LSFUN2.
                                ddx
                                   j

          If the user is not satisfied with the result (e.g. because IFAIL
          lies between 3 and 8), it is worth restarting the calculations
          from a different starting point (not the point at which the
          failure occurred) in order to avoid the region which caused the
          failure. Repeated failure may indicate some defect in the
          formulation of the problem.

          7. Accuracy

          If the problem is reasonably well scaled and a successful exit is
          made then, for a computer with a mantissa of t decimals, one
          would expect to get t/2-1 decimals accuracy in the components of
          x and between t-1 (if F(x) is of order 1 at the minimum) and 2t-2
          (if F(x) is close to zero at the minimum) decimals accuracy in
          F(x).

          8. Further Comments

          The number of iterations required depends on the number of
          variables, the number of residuals and their behaviour, and the
          distance of the starting point from the solution. The number of
          multiplications performed per iteration of E04GCF varies, but for
                                   2    3
          m>>n is approximately n*m +O(n ). In addition, each iteration
          makes at least one call of LSFUN2. So, unless the residuals and
          their derivatives can be evaluated very quickly, the run time
          will be dominated by the time spent in LSFUN2.

          Ideally the problem should be scaled so that the minimum value of
          the sum of squares is in the range (0,1) and so that at points a
          unit distance away from the solution the sum of squares is
          approximately a unit value greater than at the minimum. It is
          unlikely that the user will be able to follow these
          recommendations very closely, but it is worth trying (by
          guesswork), as sensible scaling will reduce the difficulty of the
          minimization problem, so that E04GCF will take less computer
          time.

          When the sum of squares represents the goodness of fit of a
          nonlinear model to observed data, elements of the variance-
          covariance matrix of the estimated regression coefficients can be
          computed by a subsequent call to E04YCF, using information
          returned in segments of the workspace array W. See E04YCF for
          further details.

          9. Example

          To find the least-squares estimates of x , x  and x  in the model
                                                  1   2      3

                                            t
                                             1
                                   y=x + ---------
                                      1  x t +x t
                                          2 2  3 3

          using the 15 sets of data given in the following table.

                                  y    t   t    t
                                        1   2    3
                                 0.14  1.0 15.0 1.0
                                 0.18  2.0 14.0 2.0
                                 0.22  3.0 13.0 3.0
                                 0.25  4.0 12.0 4.0
                                 0.29  5.0 11.0 5.0
                                 0.32  6.0 10.0 6.0
                                 0.35  7.0  9.0 7.0
                                 0.39  8.0  8.0 8.0
                                 0.37  9.0  7.0 7.0
                                 0.58 10.0  6.0 6.0
                                 0.73 11.0  5.0 5.0
                                 0.96 12.0  4.0 4.0
                                 1.34 13.0  3.0 3.0
                                 2.10 14.0  2.0 2.0
                                 4.39 15.0  1.0 1.0

          The program uses (0.5, 1.0, 1.5) as the initial guess at the
          position of the minimum.

          The example program is not reproduced here. The source code for
          all example programs is distributed with the NAG Foundation
          Library software and should be available on-line.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

          E04 -- Minimizing or Maximizing a Function                 E04JAF
                  E04JAF -- NAG Foundation Library Routine Document

          Note: Before using this routine, please read the Users' Note for
          your implementation to check implementation-dependent details.
          The symbol (*) after a NAG routine name denotes a routine that is
          not included in the Foundation Library.

          1. Purpose

          E04JAF is an easy-to-use quasi-Newton algorithm for finding a
          minimum of a function F(x ,x ,...,x ), subject to fixed upper and
                                   1  2      n
          lower bounds of the independent variables x ,x ,...,x , using
                                                     1  2      n
          function values only.

          It is intended for functions which are continuous and which have
          continuous first and second derivatives (although it will usually
          work even if the derivatives have occasional discontinuities).

          2. Specification

                 SUBROUTINE E04JAF (N, IBOUND, BL, BU, X, F, IW, LIW, W,
                1                   LW, IFAIL)
                 INTEGER          N, IBOUND, IW(LIW), LIW, LW, IFAIL
                 DOUBLE PRECISION BL(N), BU(N), X(N), F, W(LW)

          3. Description

          This routine is applicable to problems of the form:

            Minimize  F(x ,x ,...,x ) subject to l <=x <=u  , j=1,2,...,n
                         1  2      n              j   j   j

          when derivatives of F(x) are unavailable.

          Special provision is made for problems which actually have no
          bounds on the x , problems which have only non-negativity bounds
                         j
          and problems in which l =l =...=l  and u =u =...=u . The user
                                 1  2      n      1  2      n
          must supply a subroutine FUNCT1 to calculate the value of F(x) at
          any point x.

          From a starting point supplied by the user there is generated, on
          the basis of estimates of the gradient and the curvature of F(x),
          a sequence of feasible points which is intended to converge to a
          local minimum of the constrained function. An attempt is made to
          verify that the final point is a minimum.

          4. References

          [1]   Gill P E and Murray W (1976) Minimization subject to bounds
                on the variables. Report NAC 72. National Physical
                Laboratory.

          5. Parameters

           1:  N -- INTEGER                                           Input
               On entry: the number n of independent variables.
               Constraint: N >= 1.

           2:  IBOUND -- INTEGER                                      Input
               On entry: indicates whether the facility for dealing with
               bounds of special forms is to be used.

               It must be set to one of the following values:
               IBOUND = 0
                     if the user will be supplying all the l  and u
                                                            j      j
                     individually.

               IBOUND = 1
                     if there are no bounds on any x .
                                                    j

               IBOUND = 2
                     if all the bounds are of the form 0<=x .
                                                           j

               IBOUND = 3
                     if l =l =...=l  and u =u =...=u .
                         1  2      n      1  2      n

           3:  BL(N) -- DOUBLE PRECISION array                 Input/Output
               On entry: the lower bounds l .
                                           j

               If IBOUND is set to 0, the user must set BL(j) to l , for
                                                                  j
               j=1,2,...,n. (If a lower bound is not specified for a
                                                                          6
               particular x , the corresponding BL(j) should be set to -10.)
                           j

               If IBOUND is set to 3, the user must set BL(1) to l ; E04JAF
                                                                  1
               will then set the remaining elements of BL equal to BL(1).
               On exit: the lower bounds actually used by E04JAF.

           4:  BU(N) -- DOUBLE PRECISION array                 Input/Output
               On entry: the upper bounds u .
                                           j

               If IBOUND is set to 0, the user must set BU(j) to u , for
                                                                  j
               j=1,2,...,n. (If an upper bound is not specified for a
                                                                         6
               particular x , the corresponding BU(j) should be set to 10.)
                           j

               If IBOUND is set to 3, the user must set BU(1) to u ; E04JAF
                                                                  1
               will then set the remaining elements of BU equal to BU(1).
               On exit: the upper bounds actually used by E04JAF.

           5:  X(N) -- DOUBLE PRECISION array                  Input/Output
               On entry: X(j) must be set to an estimate of the jth
               component of the position of the minimum, for j=1,2,...,n.
               On exit: the lowest point found during the calculations.
               Thus, if IFAIL = 0 on exit, X(j) is the jth component of the
               position of the minimum.

           6:  F -- DOUBLE PRECISION                                 Output
               On exit: the value of F(x) corresponding to the final point
               stored in X.

           7:  IW(LIW) -- INTEGER array                           Workspace

           8:  LIW -- INTEGER                                         Input
               On entry: the length of IW as declared in the (sub)program
               from which E04JAF is called. Constraint: LIW >= N + 2.

           9:  W(LW) -- DOUBLE PRECISION array                    Workspace

          10:  LW -- INTEGER                                          Input
               On entry: the length of W as declared in the (sub)program
               from which E04JAF is called. Constraint: LW>=max(N*(N-
               1)/2+12*N,13).

          11:  IFAIL -- INTEGER                                Input/Output
               On entry: IFAIL must be set to 0, -1 or 1. Users who are
               unfamiliar with this parameter should refer to the Essential
               Introduction for details.

               On exit: IFAIL = 0 unless the routine detects an error or
               gives a warning (see Section 6).

               For this routine, because the values of output parameters
               may be useful even if IFAIL /=0 on exit, users are
               recommended to set IFAIL to -1 before entry. It is then
               essential to test the value of IFAIL on exit. To suppress
               the output of an error message when soft failure occurs, set
               IFAIL to 1.

          5.1. Optional Parameters

               FUNCT1 -- SUBROUTINE, supplied by the user.
                                                    External Procedure
               This routine must be supplied by the user to calculate the
               value of the function F(x) at any point x. Since this
               routine is not a parameter to E04JAF, it must be called
               FUNCT1. It should be tested separately before being used in
               conjunction with E04JAF (see the Chapter Introduction).

               Its specification is:

                      SUBROUTINE FUNCT1 (N, XC, FC)
                      INTEGER          N
                      DOUBLE PRECISION XC(N), FC

                1:  N -- INTEGER                                      Input
                    On entry: the number n of variables.

                2:  XC(N) -- DOUBLE PRECISION array                   Input
                    On entry: the point x at which the function value is
                    required.

                3:  FC -- DOUBLE PRECISION                           Output
                    On exit: the value of the function F at the current
                    point x.
               FUNCT1 must be declared as EXTERNAL in the (sub)program
               from which E04JAF is called. Parameters denoted as
               Input must not be changed by this procedure.

          6. Error Indicators and Warnings

          Errors or warnings specified by the routine:

          IFAIL= 1
               On entry N < 1,

               or       IBOUND < 0,

               or       IBOUND > 3,

               or       IBOUND = 0 and BL(j) > BU(j) for some j,

               or       IBOUND = 3 and BL(1) > BU(1),

               or       LIW < N + 2,

               or       LW<max(13,12*N+N*(N-1)/2).

          IFAIL= 2
               There have been 400*n function evaluations, yet the
               algorithm does not seem to be converging. The calculations
               can be restarted from the final point held in X. The error
               may also indicate that F(x) has no minimum.

          IFAIL= 3
               The conditions for a minimum have not all been met but a
               lower point could not be found and the algorithm has failed.

          IFAIL= 4
               An overflow has occurred during the computation. This is an
               unlikely failure, but if it occurs the user should restart
               at the latest point given in X.

          IFAIL= 5

          IFAIL= 6

          IFAIL= 7

          IFAIL= 8
               There is some doubt about whether the point x found by
               E04JAF is a minimum. The degree of confidence in the result
               decreases as IFAIL increases. Thus, when IFAIL = 5 it is
               probable that the final x gives a good estimate of the
               position of a minimum, but when IFAIL = 8 it is very
               unlikely that the routine has found a minimum.

          IFAIL= 9
               In the search for a minimum, the modulus of one of the
                                                    6
               variables has become very large  (~10 ).  This indicates
               that there is a mistake in FUNCT1, that the user's problem
               has no finite solution, or that the problem needs rescaling
               (see Section 8).

          If the user is dissatisfied with the result (e.g. because IFAIL =
          5, 6, 7 or 8), it is worth restarting the calculations from a
          different starting point (not the point at which the failure
          occurred) in order to avoid the region which caused the failure.
          If persistent trouble occurs and the gradient can be calculated,
          it may be advisable to change to a routine which uses gradients
          (see the Chapter Introduction).

          7. Accuracy

          When a successful exit is made then, for a computer with a
          mantissa of t decimals, one would expect to get about t/2-1
          decimals accuracy in x and about t-1 decimals accuracy in F,
          provided the problem is reasonably well scaled.

          8. Further Comments

          The number of iterations required depends on the number of
          variables, the behaviour of F(x) and the distance of the starting
          point from the solution. The number of operations performed in an
                                                          2
          iteration of E04JAF is roughly proportional to n . In addition,
          each iteration makes at least m+1 calls of FUNCT1, where m is the
          number of variables not fixed on bounds. So, unless F(x) can be
          evaluated very quickly, the run time will be dominated by the
          time spent in FUNCT1.

          Ideally the problem should be scaled so that at the solution the
          value of F(x) and the corresponding values of x ,x ,...,x  are
                                                         1  2      n
          each in the range (-1,+1), and so that at points a unit distance
          away from the solution, F is approximately a unit value greater
          than at the minimum. It is unlikely that the user will be able to
          follow these recommendations very closely, but it is worth trying
          (by guesswork), as sensible scaling will reduce the difficulty of
          the minimization problem, so that E04JAF will take less computer
          time.

          9. Example

          To minimize

                                2         2         4          4
                     F=(x +10x ) +5(x -x ) +(x -2x ) +10(x -x )
                         1    2      3  4     2   3       1  4

          subject to

                                      1<=x <=3
                                          1

                                      -2<=x <=0
                                           2

                                      1<=x <=3,
                                          4

          starting from the initial guess (3, - 1, 0, 1).

          The example program is not reproduced here. The source code for
          all example programs is distributed with the NAG Foundation
          Library software and should be available on-line.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

          E04 -- Minimizing or Maximizing a Function                 E04MBF
                  E04MBF -- NAG Foundation Library Routine Document

          Note: Before using this routine, please read the Users' Note for
          your implementation to check implementation-dependent details.
          The symbol (*) after a NAG routine name denotes a routine that is
          not included in the Foundation Library.

          1. Purpose

          E04MBF is an easy-to-use routine for solving linear programming
          problems, or for finding a feasible point for such problems. It
          is not intended for large sparse problems.

          2. Specification

                 SUBROUTINE E04MBF (ITMAX, MSGLVL, N, NCLIN, NCTOTL, NROWA,
                1                   A, BL, BU, CVEC, LINOBJ, X, ISTATE,
                2                   OBJLP, CLAMDA, IWORK, LIWORK, WORK,
                3                   LWORK, IFAIL)
                 INTEGER          ITMAX, MSGLVL, N, NCLIN, NCTOTL, NROWA,
                1                 ISTATE(NCTOTL), IWORK(LIWORK), LIWORK,
                2                 LWORK, IFAIL
                 DOUBLE PRECISION A(NROWA,N), BL(NCTOTL), BU(NCTOTL), CVEC
                1                 (N), X(N), OBJLP, CLAMDA(NCTOTL), WORK
                2                 (LWORK)
                 LOGICAL          LINOBJ

          3. Description

          E04MBF solves linear programming (LP) problems of the form

                                T                   (x )
                    Minimize   c x   subject to  l<=(Ax)<=u            (LP)
                             n
                    x is in R

          where c is an n element vector and A is an m by n matrix i.e.,
          there are n variables and m general linear constraints. m may be
          zero in which case the LP problem is subject only to bounds on
          the variables. Notice that upper and lower bounds are specified
          for all the variables and constraints. This form allows full
          generality in specifying other types of constraints. For example
          the ith constraint may be specified as equality by setting l =u .
                                                                      i  i
          If certain bounds are not present the associated elements of l or
          u can be set to special values that will be treated as -infty or
          +infty.

          The routine allows the linear objective function to be omitted in
          which case a feasible point for the set of constraints is sought.

          The user must supply an initial estimate of the solution.

          Users who wish to exercise additional control and users with
          problems whose solution would benefit from additional flexibility
          should consider using the comprehensive routine E04NAF.

          4. References

          [1]   Gill P E, Murray W and Wright M H (1981) Practical
                Optimization. Academic Press.

          [2]   Gill P E, Murray W, Saunders M A and Wright M H (1983)
                User's Guide for SOL/QPSOL. Report SOL 83-7. Department of
                Operations Research, Stanford University.

          5. Parameters

           1:  ITMAX -- INTEGER                                       Input
               On entry: an upper bound on the number of iterations to be
               taken. If ITMAX is not positive, then the value 50 is used
               in place of ITMAX.

           2:  MSGLVL -- INTEGER                                      Input
               On entry: indicates whether or not printout is required at
               the final solution. When printing occurs the output is on
               the advisory message channel (see X04ABF). A description of
               the printed output is given in Section 5.1. The level of
               printing is determined as follows:
               MSGLVL < 0
                     No printing.

               MSGLVL = 0
                     Printing only if an input parameter is incorrect, or
                     if the problem is so ill-conditioned that subsequent
                     overflow is likely. This setting is strongly
                     recommended in preference to MSGLVL < 0.

               MSGLVL = 1
                     Printing at the solution.

               MSGLVL > 1
                     Values greater than 1 should normally be used only at
                     the direction of NAG; such values may generate large
                     amounts of printed output.

           3:  N -- INTEGER                                           Input
               On entry: the number n of variables. Constraint: N >= 1.

           4:  NCLIN -- INTEGER                                       Input
               On entry: the number of general linear constraints in the
               problem. Constraint: NCLIN >= 0.

           5:  NCTOTL -- INTEGER                                      Input
               On entry: the value (N+NCLIN).

           6:  NROWA -- INTEGER                                       Input
               On entry:
               the first dimension of the array A as declared in the
               (sub)program from which E04MBF is called.
               Constraint: NROWA >= max(1,NCLIN).

           7:  A(NROWA,N) -- DOUBLE PRECISION array                   Input
               On entry: the leading NCLIN by n part of A must contain the
               NCLIN general constraints, with the coefficients of the ith
               constraint in the ith row of A. If NCLIN = 0, then A is not
               referenced.

           8:  BL(NCTOTL) -- DOUBLE PRECISION array                   Input
               On entry: the first n elements of BL must contain the lower
               bounds on the n variables, and when NCLIN > 0, the next
               NCLIN elements of BL must contain the lower bounds on the
               NCLIN general linear constraints. To specify a non-existent
               lower bound (l =-infty), set BL(j)<=-1.0E+20.
                             j

           9:  BU(NCTOTL) -- DOUBLE PRECISION array                   Input
               On entry: the first n elements of BU must contain the upper
               bounds on the n variables, and when NCLIN > 0, the next
               NCLIN elements of BU must contain the upper bounds on the
               NCLIN general linear constraints. To specify a non-existent
               upper bound (u =+infty), set BU(j)>=1.0E+20. Constraint:
                             j
               BL(j)<=BU(j), for j=1,2,...,NCTOTL.

          10:  CVEC(N) -- DOUBLE PRECISION array                      Input
               On entry: with LINOBJ = .TRUE., CVEC must contain the
               coefficients of the objective function. If LINOBJ = .FALSE.,
               then CVEC is not referenced.

          11:  LINOBJ -- LOGICAL                                      Input
               On entry: indicates whether or not a linear objective
               function is present. If LINOBJ = .TRUE., then the full LP
               problem is solved, but if LINOBJ = .FALSE., only a feasible
               point is found and the array CVEC is not referenced.

          12:  X(N) -- DOUBLE PRECISION array                  Input/Output
               On entry: an estimate of the solution, or of a feasible
               point. Even when LINOBJ = .TRUE. it is not necessary for the
               point supplied in X to be feasible. In the absence of better
               information all elements of X may be set to zero. On exit:
               the solution to the LP problem when LINOBJ = .TRUE., or a
               feasible point when LINOBJ = .FALSE..

               When no feasible point exists (see IFAIL = 1 in Section 6)
               then X contains the point for which the sum of the
               infeasibilities is a minimum. On return with IFAIL = 2, 3 or
               4, X contains the point at which E04MBF terminated.

          13:  ISTATE(NCTOTL) -- INTEGER array                       Output
               On exit: with IFAIL < 5, ISTATE indicates the status of
               every constraint at the final point. The first n elements of
               ISTATE refer to the upper and lower bounds on the variables
               and when NCLIN > 0 the next NCLIN elements refer to the
               general constraints.

               Their meaning is:
               ISTATE(j) Meaning

               -2        The constraint violates its lower bound. This
                         value cannot occur for any element of ISTATE when
                         a feasible point has been found.

               -1        The constraint violates its upper bound. This
                         value cannot occur for any element of ISTATE when
                         a feasible point has been found.

               0         The constraint is not in the working set (is not
                         active) at the final point. Usually this means
                         that the constraint lies strictly between its
                         bounds.

               1         This inequality constraint is in the working set
                         (is active) at its lower bound.

               2         This inequality constraint is in the working set
                         (is active) at its upper bound.

               3         This constraint is included in the working set (is
                         active) as an equality. This value can only occur
                         when BL(j) = BU(j).

          14:  OBJLP -- DOUBLE PRECISION                             Output
               On exit: when LINOBJ = .TRUE., then on successful exit,
               OBJLP contains the value of the objective function at the
               solution, and on exit with IFAIL = 2, 3 or 4, OBJLP contains
               the value of the objective function at the point returned in
               X.

               When LINOBJ = .FALSE., then on successful exit OBJLP will be
               zero and on return with IFAIL = 1, OBJLP contains the
               minimum sum of the infeasibilities corresponding to the
               point returned in X.

          15:  CLAMDA(NCTOTL) -- DOUBLE PRECISION array              Output
               On exit: when LINOBJ = .TRUE., then on successful exit, or
               on exit with IFAIL = 2, 3, or 4, CLAMDA contains the
               Lagrange multipliers (reduced costs) for each constraint
               with respect to the working set. The first n components of
               CLAMDA contain the multipliers for the bound constraints on
               the variables and the remaining NCLIN components contain the
               multipliers for the general linear constraints.

               If ISTATE(j) = 0 so that the jth constraint is not in the
               working set then CLAMDA(j) is zero. If X is optimal and
               ISTATE(j) = 1, then CLAMDA(j) should be non-negative, and if
               ISTATE(j) = 2, then CLAMDA(j) should be non-positive.

               When LINOBJ = .FALSE., all NCTOTL elements of CLAMDA are
               returned as zero.

          16:  IWORK(LIWORK) -- INTEGER array                     Workspace

          17:  LIWORK -- INTEGER                                      Input
               On entry: the length of the array IWORK as declared in the
               (sub)program from which E04MBF is called. Constraint:
               LIWORK>=2*N.

          18:  WORK(LWORK) -- DOUBLE PRECISION array              Workspace

          19:  LWORK -- INTEGER                                       Input
               On entry: the length of the array WORK as declared in the
               (sub)program from which E04MBF is called. Constraints:
               when N <= NCLIN then
                               2
                     LWORK>=2*N +6*N+4*NCLIN+NROWA;

               when 0 <= NCLIN < N then
                                       2
                     LWORK>=2*(NCLIN+1) +4*NCLIN+6*N+NROWA.

          20:  IFAIL -- INTEGER                                Input/Output
               On entry: IFAIL must be set to 0, -1 or 1. Users who are
               unfamiliar with this parameter should refer to the Essential
               Introduction for details.

               On exit: IFAIL = 0 unless the routine detects an error or
               gives a warning (see Section 6).

               For this routine, because the values of output parameters
               may be useful even if IFAIL /=0 on exit, users are
               recommended to set IFAIL to -1 before entry. It is then
               essential to test the value of IFAIL on exit. To suppress
               the output of an error message when soft failure occurs, set
               IFAIL to 1.

          5.1. Description of the Printed Output

          When MSGLVL = 1, then E04MBF will produce output on the advisory
          message channel (see X04ABF ), giving information on the final
          point. The following describes the printout associated with each
          variable.

          Output         Meaning

          VARBL          The name (V) and index j, for j=1,2,...,n, of the
                         variable.

          STATE          The state of the variable. (FR if neither bound is
                         in the working set, EQ for a fixed variable, LL if
                         on its lower bound, UL if on its upper bound and
                         TB if held on a temporary bound.) If the value of
                         the variable lies outside the upper or lower bound
                         then STATE will be ++ or -- respectively.

          VALUE          The value of the variable at the final iteration.

          LOWER BOUND    The lower bound specified for the variable.

          UPPER BOUND    The upper bound specified for the variable.

          LAGR MULT      The value of the Lagrange multiplier for the
                         associated bound.

          RESIDUAL       The difference between the value of the variable
                         and the nearer of its bounds.

          For each of the general constraints the printout is as above with
          refers to the jth element of Ax, except that VARBL is replaced
          by:

          LNCON          The name (L) and index j, for j = 1,2,...,NCLIN of
                         the constraint.

          6. Error Indicators and Warnings

          Errors or warnings specified by the routine:

          Note: when MSGLVL=1 a short description of the error is printed.

          IFAIL= 1
               No feasible point could be found. Moving violated
               constraints so that they are satisfied at the point returned
               in X gives the minimum moves necessary to make the LP
               problem feasible.

          IFAIL= 2
               The solution to the LP problem is unbounded.

          IFAIL= 3
               A total of 50 changes were made to the working set without
               altering x. Cycling is probably occurring. The user should
               consider using E04NAF with MSGLVL >= 5 to monitor constraint
               additions and deletions in order to determine whether or not
               cycling is taking place.

          IFAIL= 4
               The limit on the number of iterations has been reached.
               Increase ITMAX or consider using E04NAF to monitor progress.

          IFAIL= 5
               An input parameter is invalid. Unless MSGLVL < 0 a message
               will be printed.

          IFAILOverflow
               If the printed output before the overflow occurred contains
               a warning about serious ill-conditioning in the working set
               when adding the jth constraint, then either the user should
               try using E04NAF and experiment with the magnitude of FEATOL
               (j) in that routine, or the offending linearly dependent
               constraint (with index j) should be removed from the
               problem.

          7. Accuracy

          The routine implements a numerically stable active set strategy
          and returns solutions that are as accurate as the condition of
          the LP problem warrants on the machine.

          8. Further Comments

          The time taken by each iteration is approximately proportional to
               2      2
          min(n ,NCLIN ).

          Sensible scaling of the problem is likely to reduce the number of
          iterations required and make the problem less sensitive to
          perturbations in the data, thus improving the condition of the LP
          problem. In the absence of better information it is usually
          sensible to make the Euclidean lengths of each constraint of
          comparable magnitude. See Gill et al [1] for further information
          and advice.

          Note that the routine allows constraints to be violated by an
          absolute tolerance equal to the machine precision (see X02AJF(*))

          9. Example

          To minimize the function

                    -0.02x -0.2x -0.2x -0.2x -0.2x +0.04x +0.04x
                          1     2     3     4     5      6      7

          subject to the bounds

                                -0.01 <= x  <= 0.01
                                          1
                                -0.1 <= x  <= 0.15,
                                         2
                                -0.01 <= x  <= 0.03,
                                          3
                                -0.04 <= x  <= 0.02,
                                          4
                                -0.1 <= x  <= 0.05,
                                         5
                                -0.01 <= x
                                          6
                                -0.01 <= x
                                          7

          and the general constraints

                             x +x +x +x +x +x +x =-0.13
                              1  2  3  4  5  6  7

              0.15x +0.04x +0.02x +0.04x +0.02x +0.01x +0.03x <=-0.0049
                   1      2      3      4      5      6      7

                 0.03x +0.05x +0.08x +0.02x +0.06x +0.01x <=-0.0064
                      1      2      3      4      5      6

                     0.02x +0.04x +0.01x +0.02x +0.02x <=-0.0037
                          1      2      3      4      5

                            0.02x +0.03x +0.01x <=-0.0012
                                 1      2      5

                 -0.0992<=0.70x +0.75x +0.80x +0.75x +0.80x +0.97x
                               1      2      3      4      5      6

           -0.003<=0.02x +0.06x +0.08x +0.12x +0.02x +0.01x +0.97x <=0.002
                        1      2      3      4      5      6      7

          The initial point, which is infeasible, is

                                                                 T
                  x =(-0.01, -0.03, 0.0, -0.01, -0.1, 0.02, 0.01) .
                   0

          The example program is not reproduced here. The source code for
          all example programs is distributed with the NAG Foundation
          Library software and should be available on-line.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

          E04 -- Minimizing or Maximizing a Function                 E04NAF
                  E04NAF -- NAG Foundation Library Routine Document

          Note: Before using this routine, please read the Users' Note for
          your implementation to check implementation-dependent details.
          The symbol (*) after a NAG routine name denotes a routine that is
          not included in the Foundation Library.

          1. Purpose

          E04NAF is a comprehensive routine for solving quadratic
          programming (QP) or linear programming (LP) problems. It is not
          intended for large sparse problems.

          2. Specification

                 SUBROUTINE E04NAF (ITMAX, MSGLVL, N, NCLIN, NCTOTL, NROWA,
                1                   NROWH, NCOLH, BIGBND, A, BL, BU, CVEC,
                2                   FEATOL, HESS, QPHESS, COLD, LP, ORTHOG,
                3                   X, ISTATE, ITER, OBJ, CLAMDA, IWORK,
                4                   LIWORK, WORK, LWORK, IFAIL)
                 INTEGER          ITMAX, MSGLVL, N, NCLIN, NCTOTL, NROWA,
                1                 NROWH, NCOLH, ISTATE(NCTOTL), ITER, IWORK
                2                 (LIWORK), LIWORK, LWORK, IFAIL
                 DOUBLE PRECISION BIGBND, A(NROWA,N), BL(NCTOTL),
                1                 BU(NCTOTL), CVEC(N), FEATOL(NCTOTL), HESS
                2                 (NROWH,NCOLH), X(N), OBJ, CLAMDA(NCTOTL),
                3                 WORK(LWORK)
                 LOGICAL          COLD, LP, ORTHOG
                 EXTERNAL         QPHESS

          3. Description

          E04NAF is essentially identical to the subroutine SOL/QPSOL
          described in Gill et al [1].

          E04NAF is designed to solve the quadratic programming (QP)
          problem - the minimization of a quadratic function subject to a
          set of linear constraints on the variables. The problem is
          assumed to be stated in the following form:

                           T   1 T                    (x )
                Minimize  c x+ -x Hx   subject to  l<=(Ax)<=u ,         (1)
                               2

          where c is a constant n-vector and H is a constant n by n
          symmetric matrix; note that H is the Hessian matrix (matrix of
          second partial derivatives) of the quadratic objective function.
          The matrix A is m by n, where m may be zero; A is treated as a
          dense matrix.

          The constraints involving A will be called the general
          constraints. Note that upper and lower bounds are specified for
          all the variables and for all the general constraints. The form
          of (1) allows full generality in specifying other types of
          constraints. In particular, an equality constraint is specified
          by setting l =u . If certain bounds are not present, the
                      i  i
          associated elements of l or u can be set to special values that
          will be treated as -infty or +infty.

          The user must supply an initial estimate of the solution to (1),
          and a subroutine that computes the product Hx for any given
          vector x. If H is positive-definite or positive semi-definite,
          E04NAF will obtain a global minimum; otherwise, the solution
          obtained will be a local minimum (which may or may not be a
          global minimum). If H is defined as the zero matrix, E04NAF will
          solve the resulting linear programming (LP) problem; however,
          this can be accomplished more efficiently by setting a logical
          variable in the call of the routine (see the parameter LP in
          Section 5).

          E04NAF allows the user to provide the indices of the constraints
          that are believed to be exactly satisfied at the solution. This
          facility, known as a warm start, can lead to significant savings
          in computational effort when solving a sequence of related
          problems.

          The method has two distinct phases. In the first (the LP phase),
          an iterative procedure is carried out to determine a feasible
          point. In this context, feasibility is defined by a user-provided
          array FEATOL; the jth constraint is considered satisfied if its
          violation does not exceed FEATOL(j). The second phase (the QP
          phase) generates a sequence of feasible iterates in order to
          minimize the quadratic objective function. In both phases, a
          subset of the constraints - called the working set - is used to
          define the search direction at each iteration; typically, the
          working set includes constraints that are satisfied to within the
          corresponding tolerances in the FEATOL array.

          We now briefly describe a typical iteration in the QP phase. Let
          x  denote the estimate of the solution at the kth iteration; the
           k
          next iterate is defined by

                                 x   =x +(alpha) p
                                  k+1  k        k k

          where p  is an n-dimensional search direction and (alpha)  is a
                 k                                                 k
          scalar step length. Assume that the working (active) set contains
          t  linearly independent constraints, and let C  denote the matrix
           k                                            k
          of coefficients of the bounds and general constraints in the
          current working set.

          Let Z  denote a matrix whose columns form a basis for the null
               k
          space of C , so that C Z =0. (Note that Z  has n  columns, where
                    k           k k                k      z
                                T
          n =n-t .) The vector Z (c+Hx ) is called the projected gradient
           z    k               k     k
          at x . If the projected gradient is zero at x  (i.e., x  is a
              k                                        k         k
          constrained stationary point in the subspace defined by Z ),
                                                                   k
          Lagrange multipliers (lambda)  are defined as the solution of the
                                       k
          compatible overdetermined system

                                T
                               C (lambda) =c+Hx                         (2)
                                k        k     k

          The Lagrange multiplier (lambda) corresponding to an inequality
          constraint in the working set is said to be optimal if
          (lambda)<=0 when the associated constraint is at its upper bound,
          or if (lambda)>=0 when the associated constraint is at its lower
          bound. If a multiplier is non-optimal, the objective function can
          be reduced by deleting the corresponding constraint (with index
          JDEL, see Section 5.1) from the working set.

          If the projected gradient at x  is non-zero, the search direction
                                        k
          p  is defined as
           k

                                    p =Z p                              (3)
                                     k  k z

          where p  is an n -vector. In effect, the constraints in the
                 z        z
          working set are treated as equalities, by constraining p  to lie
                                                                  k
          within the subspace of vectors orthogonal to the rows of C . This
                                                                    k
          definition ensures that C p =0, and hence the values of the
                                   k k
          constraints in the working set are not altered by any move along
          p .
           k

          The vector p  is obtained by solving the equations
                      z

                                T        T
                               Z HZ p =-Z (c+Hx )                       (4)
                                k  k z   k     k

                       T
          (The matrix Z HZ  is called the projected Hessian matrix.) If the
                       k  k
          projected Hessian is positive-definite, the vector defined by (3)
          and (4) is the step to the minimum of the quadratic function in
          the subspace defined by Z .
                                   k

          If the projected Hessian is positive-definite and x +p  is
                                                             k  k
          feasible, (alpha)  will be taken as unity. In this case, the
                           k
          projected gradient at x    will be zero (see NORM ZTG in Section
                                 k+1
          5.1), and Lagrange multipliers can be computed (see Gill et al
          [2]). Otherwise, (alpha)  is set to the step to the 'nearest'
                                  k
          constraint (with index JADD, see Section 5.1), which is added to
          the working set at the next iteration.

          The matrix Z  is obtained from the TQ factorization of C , in
                      k                                           k
          which C  is represented as
                 k

                                   C Q=(0 T )                           (5)
                                    k      k

          where T  is reverse-triangular. It follows from (5) that Z  may
                 k                                                  k
          be taken as the first n  columns of Q. If the projected Hessian
                                 z
          is positive-definite, (3) is solved using the Cholesky
          factorization

                                      T     T
                                     Z HZ =R R
                                      k  k  k k

          where R  is upper triangular. These factorizations are updated as
                 k
          constraints enter or leave the working set (see Gill et al [2]
          for further details).

          An important feature of E04NAF is the treatment of indefiniteness
          in the projected Hessian. If the projected Hessian is positive-
          definite, it may become indefinite only when a constraint is
          deleted from the working set. In this case, a temporary
          modification (of magnitude HESS MOD, see Section 5.1) is added to
          the last diagonal element of the Cholesky factor. Once a
          modification has occurred, no further constraints are deleted
          from the working set until enough constraints have been added so
          that the projected Hessian is again positive-definite. If
          equation (1) has a finite solution, a move along the direction
          obtained by solving (4) with the modified Cholesky factor must
          encounter a constraint that is not already in the working set.

          In order to resolve indefiniteness in this way, we must ensure
          that the projected Hessian is positive-definite at the first
          iterate in the QP phase. Given the n  by n  projected Hessian, a
                                              z     z
          step-wise Cholesky factorization is performed with symmetric
          interchanges (and corresponding rearrangement of the columns of Z
          ), terminating if the next step would cause the matrix to become
          indefinite. This determines the largest possible positive-
          definite principal sub-matrix of the (permuted) projected
          Hessian. If n  steps of the Cholesky factorization have been
                       R
          successfully completed, the relevant projected Hessian is an n
                                                                        R
                                          T
          by n  positive-definite matrix Z HZ , where Z  comprises the
              R                           R  R         R
          first n  columns of Z. The quadratic function will subsequently
                 R
          be minimized within subspaces of reduced dimension until the full
          projected Hessian is positive-definite.

          If a linear program is being solved and there are fewer general
          constraints than variables, the method moves from one vertex to
          another while minimizing the objective function. When necessary,
          an initial vertex is defined by temporarily fixing some of the
          variables at their initial values.

          Several strategies are used to control ill-conditioning in the
          working set. One such strategy is associated with the FEATOL
          array. Allowing the jth constraint to be violated by as much as
          FEATOL(j) often provides a choice of constraints that could be
          added to the working set. When a choice exists, the decision is
          based on the conditioning of the working set. Negative steps are
          occasionally permitted, since x  may violate the constraint to be
                                         k
          added.

          4. References

          [1]   Gill P E, Murray W, Saunders M A and Wright M H (1983)
                User's Guide for SOL/QPSOL. Report SOL 83-7. Department of
                Operations Research, Stanford University.

          [2]   Gill P E, Murray W, Saunders M A and Wright M H (1982) The
                design and implementation of a quadratic programming
                algorithm. Report SOL 82-7. Department of Operations
                Research, Stanford University.

          [3]   Gill P E, Murray W and Wright M H (1981) Practical
                Optimization. Academic Press.

          5. Parameters

           1:  ITMAX -- INTEGER                                       Input
               On entry: an upper bound on the number of iterations to be
               taken during the LP phase or the QP phase. If ITMAX is not
               positive, then the value 50 is used in place of ITMAX.

           2:  MSGLVL -- INTEGER                                      Input
               On entry: MSGLVL must indicate the amount of intermediate
               output desired (see Section 5.1 for a description of the
               printed output). All output is written to the current
               advisory message unit (see X04ABF). For MSGLVL >= 10, each
               level includes the printout for all lower levels.
               Value   Definition

               <0      No printing.

               0       Printing only if an input parameter is incorrect, or
                       if the working set is so ill-conditioned that
                       subsequent overflow is likely. This setting is
                       strongly recommended in preference to MSGLVL < 0.

               1       The final solution only.

               5       One brief line of output for each constraint
                       addition or deletion (no printout of the final
                       solution).

               >=10    The final solution and one brief line of output for
                       each constraint addition or deletion.

               >=15    At each iteration, X, ISTATE, and the indices of the
                       free variables (i.e.,the variables not currently
                       held on a bound).

               >=20    At each iteration, the Lagrange multiplier estimates
                       and the general constraint values.

               >=30    At each iteration, the diagonal elements of the
                       matrix T associated with the TQ factorization of the
                       working set, and the diagonal elements of the
                       Cholesky factor R of the projected Hessian.

               >=80    Debug printout.

               99      The arrays CVEC and HESS.

           3:  N -- INTEGER                                           Input
               On entry: the number, n, of variables. Constraint: N >= 1.

           4:  NCLIN -- INTEGER                                       Input
               On entry: the number of general linear constraints in the
               problem. Constraint: NCLIN >= 0.

           5:  NCTOTL -- INTEGER                                      Input
               On entry: the value (N+NCLIN).

           6:  NROWA -- INTEGER                                       Input
               On entry:
               the first dimension of the array A as declared in the
               (sub)program from which E04NAF is called.
               Constraint: NROWA >= max(1,NCLIN).

           7:  NROWH -- INTEGER                                       Input
               On entry: the first dimension of the array HESS as declared
               in the (sub)program from which E04NAF is called.
               Constraint: NROWH >= 1.

           8:  NCOLH -- INTEGER                                       Input
               On entry: the column dimension of the array HESS as declared
               in the (sub)program from which E04NAF is called.
               Constraint: NCOLH >= 1.

           9:  BIGBND -- DOUBLE PRECISION                             Input
               On entry: BIGBND must denote an 'infinite' component of l
               and u. Any upper bound greater than or equal to BIGBND will
               be regarded as plus infinity, and a lower bound less than or
               equal to -BIGBND will be regarded as minus infinity.
               Constraint: BIGBND > 0.0.

          10:  A(NROWA,N) -- DOUBLE PRECISION array                   Input
               On entry: the leading NCLIN by n part of A must contain the
               NCLIN general constraints, with the ith constraint in the i
               th row of A. If NCLIN = 0, then A is not referenced.

          11:  BL(NCTOTL) -- DOUBLE PRECISION array                   Input
               On entry: the lower bounds for all the constraints, in the
               following order. The first n elements of BL must contain the
               lower bounds on the variables. If NCLIN > 0, the next NCLIN
               elements of BL must contain the lower bounds for the general
               linear constraints. To specify a non-existent lower bound
               (i.e., l =-infty), the value used must satisfy BL(j)<=-
                       j
               BIGBND To specify the jth constraint as an equality, the
               user must set BL(j) = BU(j). Constraint: BL(j) <= BU(j),
               j=1,2,...,NCTOTL.

          12:  BU(NCTOTL) -- DOUBLE PRECISION array                   Input
               On entry: the upper bounds for all the constraints, in the
               following order. The first n elements of BU must contain the
               upper bounds on the variables. If NCLIN > 0, the next NCLIN
               elements of BU must contain the upper bounds for the general
               linear constraints. To specify a non-existent upper bound
               (i.e., u =+infty), the value used must satisfy BU(j) >=
                       j
               BIGBND. To specify the jth constraint as an equality, the
               user must set BU(j) = BL(j). Constraint: BU(j) >= BL(j),
               j=1,2,...,NCTOTL.

          13:  CVEC(N) -- DOUBLE PRECISION array                      Input
               On entry: the coefficients of the linear term of the
               objective function (the vector c in equation  (1)).

          14:  FEATOL(NCTOTL) -- DOUBLE PRECISION array               Input
               On entry: a set of positive tolerances that define the
               maximum permissible absolute violation in each constraint in
               order for a point to be considered feasible, i.e., if the
               violation in constraint j is less than FEATOL(j), the point
               is considered to be feasible with respect to the jth
               constraint. The ordering of the elements of FEATOL is the
               same as that described above for BL.

               The elements of FEATOL should not be too small and a warning
               message will be printed on the current advisory message
               channel if any element of FEATOL is less than the machine
               precision (see X02AJF(*)). As the elements of FEATOL
               increase, the algorithm is less likely to encounter
               difficulties with ill-conditioning and degeneracy. However,
               larger values of FEATOL(j) mean that constraint j could be
               violated by a significant amount. It is recommended that
               FEATOL(j) be set to a value equal to the largest acceptable
               violation for constraint j. For example, if the data
               defining the constraints are of order unity and are correct
               to about 6 decimal digits, it would be appropriate to choose
                              -6
               FEATOL(j) as 10   for all relevant j. Often the square root
               of the machine precision is a reasonable choice if the
               constraint is well scaled.

          15:  HESS(NROWH,NCOLH) -- DOUBLE PRECISION array            Input
               On entry: HESS may be used to store the Hessian matrix H of
               equation (1) if desired. HESS is accessed only by the
               subroutine QPHESS and is not accessed if LP = .TRUE.. Refer
               to the specification of QPHESS (below) for further details
               of how HESS may be used to pass data to QPHESS.

          16:  QPHESS -- SUBROUTINE, supplied by the user.
                                                         External Procedure
               QPHESS must define the product of the Hessian matrix H and a
               vector x. The elements of H need not be defined explicitly.
               QPHESS is not accessed if LP is set to .TRUE. and in this
               case QPHESS may be the dummy routine E04NAN. (E04NAN is
               included in the NAG Foundation Library and so need not be
               supplied by the user. Its name may be implementation-
               dependent: see the Users' Note for your implementation for
               details.)

               Its specification is:

                      SUBROUTINE QPHESS (N, NROWH, NCOLH, JTHCOL,
                     1                   HESS, X, HX)
                      INTEGER          N, NROWH, NCOLH, JTHCOL
                      DOUBLE PRECISION HESS(NROWH,NCOLH), X(N), HX(N)

                1:  N -- INTEGER                                      Input
                    On entry: the number n of variables.

                2:  NROWH -- INTEGER                                  Input
                    On entry: the row dimension of the array HESS.

                3:  NCOLH -- INTEGER                                  Input
                    On entry: the column dimension of the array HESS.

                4:  JTHCOL -- INTEGER                                 Input
                    The input parameter JTHCOL is included to allow
                    flexibility for the user in the special situation when
                    x is the jth co-ordinate vector (i.e.,the jth column of
                    the identity matrix). This may be of interest because
                    the product Hx is then the jth column of H, which can
                    sometimes be computed very efficiently. The user may
                    code QPHESS to take advantage of this case. On entry:
                    if JTHCOL = j, where j>0, HX must contain column JTHCOL
                    of H, and hence special code may be included in QPHESS
                    to test JTHCOL if desired. However, special code is not
                    necessary, since the vector x always contains column
                    JTHCOL of the identity matrix whenever QPHESS is called
                    with JTHCOL > 0.

                5:  HESS(NROWH,NCOLH) -- DOUBLE PRECISION array       Input
                    On entry: the Hessian matrix H.

                    In some cases, it may be desirable to use a one-
                    dimensional array to transmit data or workspace to
                    QPHESS; HESS should then be declared with dimension
                    (NROWH) in the (sub)program from which E04NAF is called
                    and the parameter NCOLH must be 1.

                    In other situations, it may be desirable to compute Hx
                    without accessing HESS - for example, if H is sparse or
                    has special structure. (This is illustrated in the
                    subroutine QPHES1 in the example program in Section 9.)
                    The parameters HESS, NROWH and NCOLH may then refer to
                    any convenient array.

                    When MSGLVL = 99, the (possibly undefined) contents of
                    HESS will be printed, except if NROWH and NCOLH are
                    both 1. Also printed are the results of calling QPHESS
                    with JTHCOL = 1,2,...,n.

                6:  X(N) -- DOUBLE PRECISION array                    Input
                    On entry: the vector x.

                7:  HX(N) -- DOUBLE PRECISION array                  Output
                    On exit: HX must contain the product Hx.
               QPHESS must be declared as EXTERNAL in the (sub)program
               from which E04NAF is called. Parameters denoted as
               Input must not be changed by this procedure.

          17:  COLD -- LOGICAL                                        Input
               On entry: COLD must indicate whether the user has specified
               an initial estimate of the active set of constraints. If
               COLD is set to .TRUE., the initial working set is determined
               by E04NAF. If COLD is set to .FALSE. (a 'warm start'), the
               user must define the ISTATE array which gives the status of
               each constraint with respect to the working set. E04NAF will
               override the user's specification of ISTATE if necessary, so
               that a poor choice of working set will not cause a fatal
               error.

               The warm start option is particularly useful when E04NAF is
               called repeatedly to solve related problems.

          18:  LP -- LOGICAL                                          Input
               On entry: if LP = .FALSE., E04NAF will solve the specified
               quadratic programming problem. If LP = .TRUE., E04NAF will
               treat H as zero and solve the resulting linear programming
               problem; in this case, the parameters HESS and QPHESS will
               not be referenced.

          19:  ORTHOG -- LOGICAL                                      Input
               On entry: ORTHOG must indicate whether orthogonal
               transformations are to be used in computing and updating the
               TQ factorization of the working set
                                        A Q=(0 T),
                                         s
               where A  is a sub-matrix of A and T is reverse-triangular.
                      s
               If ORTHOG = .TRUE., the TQ factorization is computed using
               Householder reflections and plane rotations, and the matrix
               Q is orthogonal. If ORTHOG = .FALSE., stabilized elementary
               transformations are used to maintain the factorization, and
               Q is not orthogonal. A rule of thumb in making the choice is
               that orthogonal transformations require more work, but
               provide greater numerical stability. Thus, we recommend
               setting ORTHOG to .TRUE. if the problem is reasonably small
               or the active set is ill-conditioned. Otherwise, setting
               ORTHOG to .FALSE. will often lead to a reduction in solution
               time with negligible loss of reliability.

          20:  X(N) -- DOUBLE PRECISION array                  Input/Output
               On entry: an estimate of the solution. In the absence of
               better information all elements of X may be set to zero. On
               exit: from E04NAF, X contains the best estimate of the
               solution.

          21:  ISTATE(NCTOTL) -- INTEGER array                 Input/Output
               On entry: with COLD as .FALSE., ISTATE must indicate the
               status of every constraint with respect to the working set.
               The ordering of ISTATE is as follows; the first n elements
               of ISTATE refer to the upper and lower bounds on the
               variables and elements n+1 through n + NCLIN refer to the
               upper and lower bounds on Ax. The significance of each
               possible value of ISTATE(j) is as follows:
               ISTATE(j) Meaning

               -2        The constraint violates its lower bound by more
                         than FEATOL(j). This value of ISTATE cannot occur
                         after a feasible point has been found.

               -1        The constraint violates its upper bound by more
                         than FEATOL(j). This value of ISTATE cannot occur
                         after a feasible point has been found.

               0         The constraint is not in the working set. Usually,
                         this means that the constraint lies strictly
                         between its bounds.

               1         This inequality constraint is included in the
                         working set at its lower bound. The value of the
                         constraint is within FEATOL(j) of its lower bound.

               2         This inequality constraint is included in the
                         working set at its upper bound. The value of the
                         constraint is within FEATOL(j) of its upper bound.

               3         The constraint is included in the working set as
                         an equality. This value of ISTATE can occur only
                         when BL(j) = BU(j). The corresponding constraint
                         is within FEATOL(j) of its required value.
               If COLD = .TRUE., ISTATE need not be set by the user.
               However, when COLD = .FALSE., every element of ISTATE must
               be set to one of the values given above to define a
               suggested initial working set (which will be changed by
               E04NAF if necessary). The most likely values are:
               ISTATE(j) Meaning

               0         The corresponding constraint should not be in the
                         initial working set.

               1         The constraint should be in the initial working
                         set at its lower bound.

               2         The constraint should be in the initial working
                         set at its upper bound.

               3         The constraint should be in the initial working
                         set as an equality. This value must not be
                         specified unless BL(j) = BU(j). The values 1, 2 or
                         3 all have the same effect when BL(j) = BU(j).
               Note that if E04NAF has been called previously with the same
               values of N and NCLIN, ISTATE already contains satisfactory
               values. On exit: when E04NAF exits with IFAIL set to 0, 1 or
               3, the values in the array ISTATE indicate the status of the
               constraints in the active set at the solution. Otherwise,
               ISTATE indicates the composition of the working set at the
               final iterate.

          22:  ITER -- INTEGER                                       Output
               On exit: the number of iterations performed in either the LP
               phase or the QP phase, whichever was last entered.

               Note that ITER is reset to zero after the LP phase.

          23:  OBJ -- DOUBLE PRECISION                               Output
               On exit: the value of the quadratic objective function at x
               if x is feasible (IFAIL <= 5), or the sum of infeasibilities
               at x otherwise (6 <= IFAIL <= 8).

          24:  CLAMDA(NCTOTL) -- DOUBLE PRECISION array              Output
               On exit: the values of the Lagrange multiplier for each
               constraint with respect to the current working set. The
               ordering of CLAMDA is as follows; the first n components
               contain the multipliers for the bound constraints on the
               variables, and the remaining components contain the
               multipliers for the general linear constraints. If ISTATE(j)
               = 0 (i.e.,constraint j is not in the working set), CLAMDA(j)
               is zero. If x is optimal and ISTATE(j) = 1, CLAMDA(j) should
               be non-negative; if ISTATE(j) = 2, CLAMDA(j) should be non-
               positive.

          25:  IWORK(LIWORK) -- INTEGER array                     Workspace

          26:  LIWORK -- INTEGER                                      Input
               On entry:
               the dimension of the array IWORK as declared in the
               (sub)program from which E04NAF is called.
               Constraint: LIWORK>=2*N.

          27:  WORK(LWORK) -- DOUBLE PRECISION array              Workspace

          28:  LWORK -- INTEGER                                       Input
               On entry:
               the dimension of the array WORK as declared in the
               (sub)program from which E04NAF is called.
               Constrai if LP = .FALSE. or NCLIN >= N then
               nts:               2
                        LWORK>=2*N +4*N*NCLIN+NROWA.

                        if LP = .TRUE. and NCLIN < N then
                                          2
                        LWORK>=2*(NCLIN+1) +4*N+2*NCLIN+NROWA.
               If MSGLVL > 0, the amount of workspace provided and the
               amount of workspace required are output on the current
               advisory message unit (as defined by X04ABF). As an
               alternative to computing LWORK from the formula given above,
               the user may prefer to obtain an appropriate value from the
               output of a preliminary run with a positive value of MSGLVL
               and LWORK set to 1 (E04NAF will then terminate with IFAIL =
               9).

          29:  IFAIL -- INTEGER                                Input/Output
               On entry: IFAIL must be set to 0, -1 or 1. Users who are
               unfamiliar with this parameter should refer to the Essential
               Introduction for details.

               On exit: IFAIL = 0 unless the routine detects an error or
               gives a warning (see Section 6).

               For this routine, because the values of output parameters
               may be useful even if IFAIL /=0 on exit, users are
               recommended to set IFAIL to -1 before entry. It is then
               essential to test the value of IFAIL on exit. To suppress
               the output of an error message when soft failure occurs, set
               IFAIL to 1.

               IFAIL contains zero on exit if x is a strong local minimum.
               i.e., the projected gradient is neglible, the Lagrange
               multipliers are optimal, and the projected Hessian is
               positive-definite. In some cases, a zero value of IFAIL
               means that x is a global minimum (e.g. when the Hessian
               matrix is positive-definite).

          5.1. Description of the Printed Output

          When MSGLVL >= 5, a line of output is produced for every change
          in the working set (thus, several lines may be printed during a
          single iteration).

          To aid interpretation of the printed results, we mention the
          convention for numbering the constraints: indices 1 through to n
          refer to the bounds on the variables, and when NCLIN > 0 indices
          n+1 through to n + NCLIN refer to the general constraints. When
          the status of a constraint changes, the index of the constraint
          is printed, along with the designation L (lower bound), U (upper
          bound) or E (equality).

          In the LP phase, the printout includes the following:

          ITN            is the iteration count.

          JDEL           is the index of the constraint deleted from the
                         working set. If JDEL is zero, no constraint was
                         deleted.

          JADD           is the index of the constraint added to the
                         working set. If JADD is zero, no constraint was
                         added.

          STEP           is the step taken along the computed search
                         direction.

          COND T         is a lower bound on the condition number of the
                         matrix of predicted active constraints.

          NUMINF         is the number of violated constraints
                         (infeasibilities).

          SUMINF         is a weighted sum of the magnitudes of the
                         constraint violations.

                                                                        T
          LPOBJ          is the value of the linear objective function c x.
                         It is printed only if LP = .TRUE..

          During the QP phase, the printout includes the following:

          ITN            is the iteration count (reset to zero after the LP
                         phase).

          JDEL           is the index of the constraint deleted from the
                         working set. If JDEL is zero, no constraint was
                         deleted.

          JADD           is the index of the constraint added to the
                         working set. If JADD is zero, no constraint was
                         added.

          STEP           is the step (alpha)  taken along the direction of
                                            k
                         search (if STEP is 1.0, the current point is a
                         minimum in the subspace defined by the current
                         working set).

          NHESS          is the number of calls to subroutine QPHESS.

          OBJECTIVE      is the value of the quadratic objective function.

          NCOLZ          is the number of columns of Z (see Section 3). In
                         general, it is the dimension of the subspace in
                         which the quadratic is currently being minimized.

          NORM GFREE     is the Euclidean norm of the gradient of the
                         objective function with respect to the free
                         variables, i.e. variables not currently held at a
                         bound (NORM GFREE is not printed if ORTHOG = .
                         FALSE.). In some cases, the objective function and
                         gradient are updated rather than recomputed. If
                         so, this entry will be -- to indicate that the
                         gradient with respect to the free variables has
                         not been computed.

          NORM QTG       is a weighted norm of the gradient of the
                         objective function with respect to the free
                         variables (NORM QTG is not printed if ORTHOG = .
                         TRUE.). In some cases, the objective function and
                         gradient are updated rather than recomputed. If
                         so, this entry will be -- to indicate that the
                         gradient with respect to the free variables has
                         not been computed.

          NORM ZTG       is the Euclidean norm of the projected gradient
                         (see Section 3).

          COND T         is a lower bound on the condition number of the
                         matrix of constraints in the working set.

          COND ZHZ       is a lower bound on the condition number of the
                         projected Hessian matrix.

          HESS MOD       is the correction added to the diagonal of the
                         projected Hessian to ensure that a satisfactory
                         Cholesky factorization exists (see Section 3).
                         When the projected Hessian is sufficiently
                         positive-definite, HESS MOD will be zero.

          When MSGLVL = 1 or MSGLVL >= 10, the summary printout at the end
          of execution of E04NAF includes a listing of the status of every
          constraint. Note that default names are assigned to all variables
          and constraints.

          The following describes the printout for each variable.

          VARBL          is the name (V) and index j, j=1,2,...,n, of the
                         variable.

          STATE          gives the state of the variable (FR if neither
                         bound is in the working set, EQ if a fixed
                         variable, LL if on its lower bound, UL if on its
                         upper bound, TB if held on a temporary bound). If
                         VALUE lies outside the upper or lower bounds by
                         more than FEATOL(j), STATE will be ++ or --
                         respectively.

          VALUE          is the value of the variable at the final
                         iteration.

          LOWER BOUND    is the lower bound specified for the variable.

          UPPER BOUND    is the upper bound specified for the variable.

          LAGR MULT      is the value of the Lagrange multiplier for the
                         associated bound constraint. This will be zero if
                         STATE is FR. If x is optimal and STATE is LL, the
                         multiplier should be non-negative; if STATE is UL,
                         the multiplier should be non-positive.

          RESIDUAL       is the difference between the variable and the
                         nearer of its bounds BL(j) and BU(j).

          For each of the general constraints the printout is as above with
          refers to the jth element of Ax, except that VARBL is replaced by

          LNCON          The name (L) and index j, j=1,2,...,NCLIN, of the
                         constraint.

          6. Error Indicators and Warnings

          Errors or warnings specified by the routine:

          IFAIL= 1
               x is a weak local minimum (the projected gradient is
               negligible, the Lagrange multipliers are optimal, but the
               projected Hessian is only semi-definite). This means that
               the solution is not unique.

          IFAIL= 2
               The solution appears to be unbounded, i.e., the quadratic
               function is unbounded below in the feasible region. This
               value of IFAIL occurs when a step of infinity would have to
               be taken in order to continue the algorithm.

          IFAIL= 3
               x appears to be a local minimum, but optimality cannot be
               verified because some of the Lagrange multipliers are very
               small in magnitude.

               E04NAF has probably found a solution. However, the presence
               of very small Lagrange multipliers means that the predicted
               active set may be incorrect, or that x may be only a
               constrained stationary point rather than a local minimum.
               The method in E04NAF is not guaranteed to find the correct
               active set when there are very small multipliers. E04NAF
               attempts to delete constraints with zero multipliers, but
               this does not necessarily resolve the issue. The
               determination of the correct active set is a combinatorial
               problem that may require an extremely large amount of time.
               The occurrence of small multipliers often (but not always)
               indicates that there are redundant constraints.

          IFAIL= 4
               The iterates of the QP phase could be cycling, since a total
               of 50 changes were made to the working set without altering
               x.

               This value will occur if 50 iterations are performed in the
               QP phase without changing x. The user should check the
               printed output for a repeated pattern of constraint
               deletions and additions. If a sequence of constraint changes
               is being repeated, the iterates are probably cycling.
               (E04NAF  does not contain a method that is guaranteed to
               avoid cycling, which would be combinatorial in nature.)
               Cycling may occur in two circumstances: at a constrained
               stationary point where there are some small or zero Lagrange
               multipliers (see the discussion of IFAIL = 3); or at a point
               (usually a vertex) where the constraints that are satisfied
               exactly are nearly linearly dependent. In the latter case,
               the user has the option of identifying the offending
               dependent constraints and removing them from the problem, or
               restarting the run with larger values of FEATOL for nearly
               dependent constraints. If E04NAF terminates with IFAIL = 4,
               but no suspicious pattern of constraint changes can be
               observed, it may be worthwhile to restart with the final x
               (with or without the warm start option).

          IFAIL= 5
               The limit of ITMAX iterations was reached in the QP phase
               before normal termination occurred.

               The value of ITMAX may be too small. If the method appears
               to be making progress (e.g. the objective function is being
               satisfactorily reduced), increase ITMAX and rerun E04NAF
               (possibly using the warm start facility to specify the
               initial working set). If ITMAX is already large, but some of
               the constraints could be nearly linearly dependent, check
               the output for a repeated pattern of constraints entering
               and leaving the working set. (Near-dependencies are often
               indicated by wide variations in size in the diagonal
               elements of the T matrix, which will be printed if MSGLVL >=
               30.) In this case, the algorithm could be cycling (see the
               comments for IFAIL = 4).

          IFAIL= 6
               The LP phase terminated without finding a feasible point,
               and hence it is not possible to satisfy all the constraints
               to within the tolerances specified by the FEATOL array. In
               this case, the final iterate will reveal values for which
               there will be a feasible point (e.g. a feasible point will
               exist if the feasibility tolerance for each violated
               constraint exceeds its RESIDUAL at the final point). The
               modified problem (with altered values in FEATOL) may then be
               solved using a warm start.

               The user should check that there are no constraint
               redundancies. If the data for the jth constraint are
               accurate only to the absolute precision (delta), the user
               should ensure that the value of FEATOL(j) is greater than
               (delta). For example, if all elements of A are of order
               unity and are accurate only to three decimal places, every
                                                        -3
               component of FEATOL should be at least 10  .

          IFAIL= 7
               The iterates may be cycling during the LP phase; see the
               comments above under IFAIL = 4.

          IFAIL= 8
               The limit of ITMAX iterations was reached during the LP
               phase. See comments above under IFAIL = 5.

          IFAIL= 9
               An input parameter is invalid.

          Overflow
               If the printed output before the overflow error contains a
               warning about serious ill-conditioning in the working set
               when adding the jth constraint, it may be possible to avoid
               the difficulty by increasing the magnitude of FEATOL(j) and
               rerunning the program. If the message recurs even after this
               change, the offending linearly dependent constraint (with
               index j) must be removed from the problem. If a warning
               message did not precede the fatal overflow, the user should
               contact NAG.

          7. Accuracy

          The routine implements a numerically stable active set strategy
          and returns solutions that are as accurate as the condition of
          the QP problem warrants on the machine.

          8. Further Comments

          The number of iterations depends upon factors such as the number
          of variables and the distances of the starting point from the
          solution. The number of operations performed per iteration is
                                         2
          roughly proportional to (NFREE) , where NFREE (NFREE<=n) is the
          number of variables fixed on their upper or lower bounds.

          Sensible scaling of the problem is likely to reduce the number of
          iterations required and make the problem less sensitive to
          perturbations in the data, thus improving the condition of the QP
          problem. See the Chapter Introduction and Gill et al [1] for
          further information and advice.

          9. Example

                                    T   1 T
          To minimize the function c x+ -x Hx, where
                                        2

                                                             T
                      c=[-0.02,-0.2,-0.2,-0.2,-0.2,0.04,0.04]

                                   [2 0 0 0 0  0  0]
                                   [0 2 0 0 0  0  0]
                                   [0 0 2 2 0  0  0]
                                 H=[0 0 2 2 0  0  0]
                                   [0 0 0 0 2  0  0]
                                   [0 0 0 0 0 -2 -2]
                                   [0 0 0 0 0 -2 -2]

          subject to the bounds

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
   
        E04 -- Minimizing or Maximizing a Function                 E04UCF
                E04UCF -- NAG Foundation Library Routine Document
   
        Note: Before using this routine, please read the Users' Note for
        your implementation to check implementation-dependent details.
        The symbol (*) after a NAG routine name denotes a routine that is
        not included in the Foundation Library.
   
        Note for users via the AXIOM system: the interface to this routine
        has been enhanced for use with AXIOM and is slightly different to
        that offered in the standard version of the Foundation Library.  In
        particular, the optional parameters of the NAG routine are now
        included in the parameter list.  These are described in section
        5.1.2, below.
   
        1. Purpose
   
        E04UCF is designed to minimize an arbitrary smooth function
        subject to constraints, which may include simple bounds on the
        variables, linear constraints and smooth nonlinear constraints.
        (E04UCF  may be used for unconstrained, bound-constrained and
        linearly constrained optimization.) The user must provide
        subroutines that define the objective and constraint functions
        and as many of their first partial derivatives as possible.
        Unspecified derivatives are approximated by finite differences.
        All matrices are treated as dense, and hence E04UCF is not
        intended for large sparse problems.
   
        E04UCF uses a sequential quadratic programming (SQP) algorithm in
        which the search direction is the solution of a quadratic
        programming (QP) problem. The algorithm treats bounds, linear
        constraints and nonlinear constraints separately.
   
        2. Specification
   
               SUBROUTINE E04UCF (N, NCLIN, NCNLN, NROWA, NROWJ, NROWR,
              1                   A, BL, BU, CONFUN, OBJFUN, ITER,
              2                   ISTATE, C, CJAC, CLAMDA, OBJF, OBJGRD,
              3                   R, X, IWORK, LIWORK, WORK, LWORK,
              4                   IUSER, USER, STA, CRA, DER, FEA, FUN,
              5                   HES, INFB, INFS, LINF, LINT, LIST,
              6                   MAJI, MAJP, MINI, MINP, MON, NONF,
              7                   OPT, STE, STAO, STAC, STOO, STOC, VE,
              8                   IFAIL)
               INTEGER          N, NCLIN, NCNLN, NROWA, NROWJ, NROWR,
              1                 ITER, ISTATE(N+NCLIN+NCNLN), IWORK(LIWORK)
              2                 , LIWORK, LWORK, IUSER(*), DER, MAJI,
              3                 MAJP, MINI, MINP, MON, STAO, STAC, STOO,
              4                 STOC, VE, IFAIL
               DOUBLE PRECISION A(NROWA,*), BL(N+NCLIN+NCNLN), BU
              1                 (N+NCLIN+NCNLN), C(*), CJAC(NROWJ,*),
              2                 CLAMDA(N+NCLIN+NCNLN), OBJF, OBJGRD(N), R
              3                 (NROWR,N), X(N), WORK(LWORK), USER(*),
              4                 CRA, FEA, FUN,  INFB, INFS, LINF, LINT,
              5                 NONF, OPT, STE
               LOGICAL          LIST, STA, HES
               EXTERNAL         CONFUN, OBJFUN
   
        3. Description
   
        E04UCF is designed to solve the nonlinear programming problem --
        the minimization of a smooth nonlinear function subject to a set
        of constraints on the variables. The problem is assumed to be
        stated in the following form:
   
                                                   { x  }
               Minimize      F(x)   subject to  l<={A x }<=u,         (1)
                        n                          { L  }
               x is in R                           {c(x)}
   
        where F(x), the objective function, is a nonlinear function, A
                                                                      L
        is an n  by n constant matrix, and c(x) is an n  element vector
               L                                       N
        of nonlinear constraint functions. (The matrix A  and the vector
                                                        L
        c(x) may be empty.) The objective function and the constraint
        functions are assumed to be smooth, i.e., at least twice-
        continuously differentiable. (The method of E04UCF will usually
        solve (1) if there are only isolated discontinuities away from
        the solution.)
   
        This routine is essentially identical to the subroutine SOL/NPSOL
        described in Gill et al [8].
   
        Note that upper and lower bounds are specified for all the
        variables and for all the constraints.
   
        An equality constraint can be specified by setting l =u . If
                                                            i  i
        certain bounds are not present, the associated elements of l or u
        can be set to special values that will be treated as -infty or
        +infty.
   
        If there are no nonlinear constraints in (1) and F is linear or
        quadratic then one of E04MBF, E04NAF or E04NCF(*) will generally
        be more efficient. If the problem is large and sparse the MINOS
        package (see Murtagh and Saunders [13]) should be used, since
        E04UCF treats all matrices as dense.
   
        The user must supply an initial estimate of the solution to (1),
        together with subroutines that define F(x), c(x) and as many
        first partial derivatives as possible; unspecified derivatives
        are approximated by finite differences.
   
        The objective function is defined by subroutine OBJFUN, and the
        nonlinear constraints are defined by subroutine CONFUN. On every
        call, these subroutines must return appropriate values of the
        objective and nonlinear constraints. The user should also provide
        the available partial derivatives. Any unspecified derivatives
        are approximated by finite differences; see Section 5.1 for a
        discussion of the optional parameter Derivative Level. Just
        before either OBJFUN or CONFUN is called, each element of the
        current gradient array OBJGRD or CJAC is initialised to a special
        value. On exit, any element that retains the value is estimated
        by finite differences. Note that if there are nonlinear
        costraints, then the first call to CONFUN will precede the first
        call to OBJFUN.
   
        For maximum reliability, it is preferable for the user to provide
        all partial derivatives (see Chapter 8 of Gill et al [10], for a
        detailed discussion). If all gradients cannot be provided, it is
        similarly advisable to provide as many as possible. While
        developing the subroutines OBJFUN and CONFUN, the optional
        parameter Verify (see Section 5.1) should be used to check the
        calculation of any known gradients.
   
        E04UCF implements a sequential quadratic programming (SQP)
        method. The document for E04NCF(*) should be consulted in
        conjunction with this document.
   
        In the rest of this section we briefly summarize the main
        features of the method of E04UCF. Where possible, explicit
        reference is made to the names of variables that are parameters
        of subroutines E04UCF or appear in the printed output (see
        Section 5.2).
   
        At a solution of (1), some of the constraints will be active,
        i.e., satisfied exactly. An active simple bound constraint
        implies that the corresponding variable is fixed at its bound,
        and hence the variables are partitioned into fixed and free
        variables. Let C denote the m by n matrix of gradients of the
        active general linear and nonlinear constraints. The number of
        fixed variables will be denoted by n  , with n   (n  =n-n  ) the
                                            FX        FR   FR    FX
        number of free variables. The subscripts 'FX' and 'FR' on a
        vector or matrix will denote the vector or matrix composed of the
        components corresponding to fixed or free variables.
   
        A point x is a first-order Kuhn-Tucker point for (1) (see, e.g.,
        Powell [14]) if the following conditions hold:
   
             (i) x is feasible;
   
             (ii) there exist vectors (xi) and (lambda) (the Lagrange
             multiplier vectors for the bound and general constraints)
             such that
                                  T
                               g=C (lambda)+(xi),                    (2)
             where g is the gradient of F evaluated at x, and (xi) =0 if
                                                                  j
             the jth variable is free.
   
             (iii) The Lagrange multiplier corresponding to an
             inequality constraint active at its lower bound must be
             non-negative, and non-positive for an inequality constraint
             active at its upper bound.
   
        Let Z denote a matrix whose columns form a basis for the set of
        vectors orthogonal to the rows of C  ; i.e., C  Z=0. An
                                           FR         FR
        equivalent statement of the condition (2) in terms of Z is
   
                                     T
                                    Z g  =0.
                                       FR
   
                    T
        The vector Z g   is termed the projected gradient of F at x.
                      FR
        Certain additional conditions must be satisfied in order for a
        first-order Kuhn-Tucker point to be a solution of (1) (see, e.g.,
        Powell [14]).
   
        The method of E04UCF is a sequential quadratic programming (SQP)
        method. For an overview of SQP methods, see, for example,
        Fletcher [5], Gill et al [10] and Powell [15].
   
        The basic structure of E04UCF involves major and minor
        iterations. The major iterations generate a sequence of iterates
                               *
        {x } that converge to x , a first-order Kuhn-Tucker point of (1).
          k
                                                      _
        At a typical major iteration, the new iterate x is defined by
   
                                _
                                x=x+(alpha)p                          (3)
   
        where x is the current iterate, the non-negative scalar (alpha)
        is the step length, and p is the search direction. (For
        simplicity, we shall always consider a typical iteration and
        avoid reference to the index of the iteration.) Also associated
        with each major iteration are estimates of the Lagrange
        multipliers and a prediction of the active set.
   
        The search direction p in (3) is the solution of a quadratic
        programming subproblem of the form
   
                         T   1 T                  _  { p }  _
              Minimize  g p+ -p Hp,   subject to  l<={A p}<=u,        (4)
               p             2                       { L }
                                                     {A p}
                                                     { N }
   
        where g is the gradient of F at x, the matrix H is a positive-
        definite quasi-Newton approximation to the Hessian of the
        Lagrangian function (see Section 8.3), and A  is the Jacobian
                                                    N
        matrix of c evaluated at x. (Finite-difference estimates may be
        used for g and A ; see the optional parameter Derivative Level in
                        N
        Section 5.1.) Let l in (1) be partitioned into three sections:
        l , l  and l , corresponding to the bound, linear and nonlinear
         B   L      N
                                _
        constraints. The vector l in (4) is similarly partitioned, and is
        defined as
   
                       _         _               _
                       l =l -x,  l =l -A x,  and l =l -c,
                        B  B      L  L  L         N  N
   
        where c is the vector of nonlinear constraints evaluated at x.
                   _
        The vector u is defined in an analogous fashion.
   
        The estimated Lagrange multipliers at each major iteration are
        the Lagrange multipliers from the subproblem (4) (and similarly
        for the predicted active set). (The numbers of bounds, general
        linear and nonlinear constraints in the QP active set are the
        quantities Bnd, Lin and Nln in the printed output of E04UCF.) In
        E04UCF, (4) is solved using E04NCF(*). Since solving a quadratic
        program as an iterative procedure, the minor iterations of E04UCF
        are the iterations of E04NCF(*). (More details about solving the
        subproblem are given in Section 8.1.)
   
        Certain matrices associated with the QP subproblem are relevant
        in the major iterations. Let the subscripts 'FX' and 'FR' refer
        to the predicted fixed and free variables, and let C denote the m
        by n matrix of gradients of the general linear and nonlinear
        constraints in the predicted active set. First, we have available
        the TQ factorization of C  :
                                 FR
   
                               C  Q  =(0 T),                          (5)
                                FR FR
   
        where T is a nonsingular m by m reverse-triangular matrix (i.e.,
        t  =0 if i+j<m), and the non-singular n   by n   matrix Q   is
         ij                                    FR     FR         FR
        the product of orthogonal transformations (see Gill et al [6]).
        Second, we have the upper-triangular Cholesky factor R of the
        transformed and re-ordered Hessian matrix
   
                                T       T~
                               R R=H ==Q HQ,                          (6)
                                    Q
   
              ~
        where H is the Hessian H with rows and columns permuted so that
        the free variables are first, and Q is the n by n matrix
   
                                   (Q    )
                                   ( FR  )
                                 Q=(  I  ),                           (7)
                                   (   FX)
   
        with I   the identity matrix of order n  . If the columns of Q
              FX                               FX                     FR
        are partitioned so that
   
                                   Q  =(Z Y),
                                    FR
   
        the n (n ==n  -m) columns of Z form a basis for the null space of
             z  z   FR
                                                                     T
        C  . The matrix Z is used to compute the projected gradient Z g
         FR                                                            FR
        at the current iterate. (The values Nz, Norm Gf and Norm Gz
                                                            T
        printed by E04UCF give n  and the norms of g   and Z g  .)
                                z                   FR        FR
   
        A theoretical characteristic of SQP methods is that the predicted
        active set from the QP subproblem (4) is identical to the correct
                                          *
        active set in a neighbourhood of x . In E04UCF, this feature is
        exploited by using the QP active set from the previous iteration
        as a prediction of the active set for the next QP subproblem,
        which leads in practice to optimality of the subproblems in only
        one iteration as the solution is approached. Separate treatment
        of bound and linear constraints in E04UCF also saves computation
        in factorizing C   and H .
                        FR      Q
   
        Once p has been computed, the major iteration proceeds by
        determining a step length (alpha) that produces a 'sufficient
        decrease' in an augmented Lagrangian merit function (see Section
        8.2). Finally, the approximation to the transformed Hessian
        matrix H  is updated using a modified BFGS quasi-Newton update
                Q
        (see Section 8.3) to incorporate new curvature information
                                       _
        obtained in the move from x to x.
   
        On entry to E04UCF, an iterative procedure from E04NCF(*) is
        executed, starting with the user-provided initial point, to find
        a point that is feasible with respect to the bounds and linear
        constraints (using the tolerance specified by Linear Feasibility
        Tolerance see Section 5.1). If no feasible point exists for the
        bound and linear constraints, (1) has no solution and E04UCF
        terminates. Otherwise, the problem functions will thereafter be
        evaluated only at points that are feasible with respect to the
        bounds and linear constraints. The only exception involves
        variables whose bounds differ by an amount comparable to the
        finite-difference interval (see the discussion of Difference
        Interval in Section 5.1). In contrast to the bounds and linear
        constraints, it must be emphasised that the nonlinear constraints
        will not generally be satisfied until an optimal point is
        reached.
   
        Facilities are provided to check whether the user-provided
        gradients appear to be correct (see the optional parameter Verify
        in Section 5.1). In general, the check is provided at the first
        point that is feasible with respect to the linear constraints and
        bounds. However, the user may request that the check be performed
        at the initial point.
   
        In summary, the method of E04UCF first determines a point that
        satisfies the bound and linear constraints. Thereafter, each
        iteration includes:
   
        (a)   the solution of a quadratic programming subproblem;
   
        (b)   a linesearch with an augmented Lagrangian merit function;
              and
   
        (c)   a quasi-Newton update of the approximate Hessian of the
              Lagrangian function.
   
        These three procedures are described in more detail in Section 8.
   
        4. References
   
        [1]   Dennis J E Jr and More J J (1977) Quasi-Newton Methods,
              Motivation and Theory. SIAM Review. 19 46--89.
   
        [2]   Dennis J E Jr and Schnabel R B (1981) A New Derivation of
              Symmetric Positive-Definite Secant Updates. Nonlinear
              Programming 4. (ed O L Mangasarian, R R Meyer and S M.
              Robinson) Academic Press. 167--199.
   
        [3]   Dennis J E Jr and Schnabel R B (1983) Numerical Methods for
              Unconstrained Optimixation and Nonlinear Equations.
              Prentice-Hall.
   
        [4]   Dongarra J J, Du Croz J J, Hammarling S and Hanson R J
              (1985) A Proposal for an Extended set of Fortran Basic
              Linear Algebra Subprograms. SIGNUM Newsletter. 20 (1) 2--18.
   
        [5]   Fletcher R (1981) Practical Methods of Optimization, Vol 2.
              Constrained Optimization. Wiley.
   
        [6]   Gill P E, Murray W, Saunders M A and Wright M H (1984)
              User's Guide for SOL/QPSOL Version 3.2. Report SOL 84-5.
              Department of Operations Research, Stanford University.
   
        [7]   Gill P E, Murray W, Saunders M A and Wright M H (1984)
              Procedures for Optimization Problems with a Mixture of
              Bounds and General Linear Constraints. ACM Trans. Math.
              Softw. 10 282--298.
   
        [8]   Gill P E, Hammarling S, Murray W, Saunders M A and Wright M
              H (1986) User's Guide for LSSOL (Version 1.0). Report SOL
              86-1. Department of Operations Research, Stanford
              University.
   
        [9]   Gill P E, Murray W, Saunders M A and Wright M H (1986) Some
              Theoretical Properties of an Augmented Lagrangian Merit
              Function. Report SOL 86-6R. Department of Operations
              Research, Stanford University.
   
        [10]  Gill P E, Murray W and Wright M H (1981) Practical
              Optimization. Academic Press.
   
        [11]  Hock W and Schittkowski K (1981) Test Examples for Nonlinear
              Programming Codes. Lecture Notes in Economics and
              Mathematical Systems. 187 Springer-Verlag.
   
        [12]  Lawson C L, Hanson R J, Kincaid D R and Krogh F T (1979)
              Basic Linear Algebra Subprograms for Fortran Usage. ACM
              Trans. Math. Softw. 5 308--325.
   
        [13]  Murtagh B A and Saunders M A (1983) MINOS 5.0 User's Guide.
              Report SOL 83-20. Department of Operations Research,
              Stanford University.
   
        [14]  Powell M J D (1974) Introduction to Constrained
              Optimization. Numerical Methods for Constrained
              Optimization. (ed P E Gill and W Murray) Academic Press. 1--
              28.
   
        [15]  Powell M J D (1983) Variable Metric Methods in Constrained
              Optimization. Mathematical Programming: The State of the
              Art. (ed A Bachem, M Groetschel and B Korte) Springer-
              Verlag. 288--311.
   
        5. Parameters
   
         1:  N -- INTEGER                                           Input
             On entry: the number, n, of variables in the problem.
             Constraint: N > 0.
   
         2:  NCLIN -- INTEGER                                       Input
             On entry: the number, n , of general linear constraints in
                                    L
             the problem. Constraint: NCLIN >= 0.
   
         3:  NCNLN -- INTEGER                                       Input
             On entry: the number, n , of nonlinear constraints in the
                                    N
             problem. Constraint: NCNLN >= 0.
   
         4:  NROWA -- INTEGER                                       Input
             On entry:
             the first dimension of the array A as declared in the
             (sub)program from which E04UCF is called.
             Constraint: NROWA >= max(1,NCLIN).
   
         5:  NROWJ -- INTEGER                                       Input
             On entry:
             the first dimension of the array CJAC as declared in the
             (sub)program from which E04UCF is called.
             Constraint: NROWJ >= max(1,NCNLN).
   
         6:  NROWR -- INTEGER                                       Input
             On entry:
             the first dimension of the array R as declared in the
             (sub)program from which E04UCF is called.
             Constraint: NROWR >= N.
   
         7:  A(NROWA,*) -- DOUBLE PRECISION array                   Input
             The second dimension of the array A must be >= N for NCLIN >
             0. On entry: the ith row of the array A must contain the ith
             row of the matrix A  of general linear constraints in (1).
                                L
             That is, the ith row contains the coefficients of the ith
             general linear constraint, for i = 1,2,...,NCLIN.
   
             If NCLIN = 0 then the array A is not referenced.
   
         8:  BL(N+NCLIN+NCNLN) -- DOUBLE PRECISION array            Input
             On entry: the lower bounds for all the constraints, in the
             following order. The first n elements of BL must contain the
             lower bounds on the variables. If NCLIN > 0, the next n
                                                                    L
             elements of BL must contain the lower bounds on the general
             linear constraints. If NCNLN > 0, the next n  elements of BL
                                                         N
             must contain the lower bounds for the nonlinear constraints.
             To specify a non-existent lower bound (i.e., l =-infty), the
                                                           j
             value used must satisfy BL(j)<=-BIGBND, where BIGBND is the
             value of the optional parameter Infinite Bound Size whose
                                10
             default value is 10   (see Section 5.1). To specify the jth
             constraint as an equality, the user must set BL(j) = BU(j) =
             (beta), say, where |(beta)|<BIGBND. Constraint: BL(j) <= BU(
             j), for j=1,2,...,N+NCLIN+NCNLN.
   
         9:  BU(N+NCLIN+NCNLN) -- DOUBLE PRECISION array            Input
             On entry: the upper bounds for all the constraints in the
             following order. The first n elements of BU must contain the
             upper bounds on the variables. If NCLIN > 0, the next n
                                                                    L
             elements of BU must contain the upper bounds on the general
             linear constraints. If NCNLN > 0, the next n  elements of BU
                                                         N
             must contain the upper bounds for the nonlinear constraints.
             To specify a non-existent upper bound (i.e., u =+infty), the
                                                           j
             value used must satisfy BU(j) >= BIGBND, where BIGBND is the
             value of the optional parameter Infinite Bound Size, whose
                                10
             default value is 10   (see Section 5.1). To specify the jth
             constraint as an equality, the user must set BU(j) = BL(j) =
             (beta), say, where |(beta)| < BIGBND. Constraint: BU(j) >=
             BL(j), for j=1,2,...,N+NCLIN+NCNLN.
   
        10:  CONFUN -- SUBROUTINE, supplied by the user.
                                                       External Procedure
             CONFUN must calculate the vector c(x) of nonlinear
             constraint functions and (optionally) its Jacobian for a
             specified n element vector x. If there are no nonlinear
             constraints (NCNLN=0), CONFUN will never be called by E04UCF
             and CONFUN may be the dummy routine E04UDM. (E04UDM is
             included in the NAG Foundation Library and so need not be
             supplied by the user. Its name may be implementation-
             dependent: see the Users' Note for your implementation for
             details.) If there are nonlinear constraints, the first call
             to CONFUN will occur before the first call to OBJFUN.
   
             Its specification is:
   
                    SUBROUTINE CONFUN (MODE, NCNLN, N, NROWJ, NEEDC,
                   1                   X, C, CJAC, NSTATE, IUSER,
                   2                   USER)
                    INTEGER          MODE, NCNLN, N, NROWJ, NEEDC
                   1                 (NCNLN), NSTATE, IUSER(*)
                    DOUBLE PRECISION X(N), C(NCNLN), CJAC(NROWJ,N),
                   1                 USER(*)
   
              1:  MODE -- INTEGER                            Input/Output
                  On entry: MODE indicates the values that must be
                  assigned during each call of CONFUN. MODE will always
                  have the value 2 if all elements of the Jacobian are
                  available, i.e., if Derivative Level is either 2 or 3
                  (see Section 5.1). If some elements of CJAC are
                  unspecified, E04UCF will call CONFUN with MODE = 0, 1,
                  or 2:
   
                  If MODE = 2, only the elements of C corresponding to
                  positive values of NEEDC must be set (and similarly for
                  the available components of the rows of CJAC).
   
                  If MODE = 1, the available components of the rows of
                  CJAC corresponding to positive values in NEEDC must be
                  set. Other rows of CJAC and the array C will be
                  ignored.
   
                  If MODE = 0, the components of C corresponding to
                  positive values in NEEDC must be set. Other components
                  and the array CJAC are ignored. On exit: MODE may be
                  set to a negative value if the user wishes to terminate
                  the solution to the current problem. If MODE is
                  negative on exit from CONFUN then E04UCF will terminate
                  with IFAIL set to MODE.
   
              2:  NCNLN -- INTEGER                                  Input
                  On entry: the number, n , of nonlinear constraints.
                                         N
   
              3:  N -- INTEGER                                      Input
                  On entry: the number, n, of variables.
   
              4:  NROWJ -- INTEGER                                  Input
                  On entry: the first dimension of the array CJAC.
   
              5:  NEEDC(NCNLN) -- INTEGER array                     Input
                  On entry: the indices of the elements of C or CJAC that
                  must be evaluated by CONFUN. If NEEDC(i)>0 then the ith
                  element of C and/or the ith row of CJAC (see parameter
                  MODE above) must be evaluated at x.
   
              6:  X(N) -- DOUBLE PRECISION array                    Input
                  On entry: the vector x of variables at which the
                  constraint functions are to be evaluated.
   
              7:  C(NCNLN) -- DOUBLE PRECISION array               Output
                  On exit: if NEEDC(i)>0 and MODE = 0 or 2, C(i) must
                  contain the value of the ith constraint at x. The
                  remaining components of C, corresponding to the non-
                  positive elements of NEEDC, are ignored.
   
              8:  CJAC(NROWJ,N) -- DOUBLE PRECISION array          Output
                  On exit: if NEEDC(i)>0 and MODE = 1 or 2, the ith row
                  of CJAC must contain the available components of the
                  vector (nabla)c  given by
                                 i
                                      ( ddc   ddc       ddc )
                                      (    i     i         i)T
                            (nabla)c =( ----, ----,..., ----) ,
                                    i ( ddx   ddx       ddx )
                                      (    1     2         n)
                         ddc
                            i
                  where  ---- is the partial derivative of the ith
                         ddx
                            j
                  constraint with respect to the jth variable, evaluated
                  at the point x. See also the parameter NSTATE below.
                  The remaining rows of CJAC, corresponding to non-
                  positive elements of NEEDC, are ignored.
   
                  If all constraint gradients (Jacobian elements) are
                  known (i.e., Derivative Level = 2 or 3; see Section 5.1)
                  any constant elements may be assigned to CJAC one
                  time only at the start of the optimization. An element
                  of CJAC that is not subsequently assigned in CONFUN
                  will retain its initial value throughout. Constant
                  elements may be loaded into CJAC either before the call
                  to E04UCF or during the first call to CONFUN (signalled
                  by the value NSTATE = 1). The ability to preload
                  constants is useful when many Jacobian elements are
                  identically zero, in which case CJAC may be initialised
                  to zero and non-zero elements may be reset by CONFUN.
   
                  Note that constant non-zero elements do affect the
                  values of the constraints. Thus, if CJAC(i,j) is set to
                  a constant value, it need not be reset in subsequent
                  calls to CONFUN, but the value CJAC(i,j)*X(j) must
                  nonetheless be added to C(i).
   
                  It must be emphasized that, if Derivative Level < 2,
                  unassigned elements of CJAC are not treated as
                  constant; they are estimated by finite differences, at
                  non-trivial expense. If the user does not supply a
                  value for Difference Interval (see Section 5.1), an
                  interval for each component of x is computed
                  automatically at the start of the optimization. The
                  automatic procedure can usually identify constant
                  elements of CJAC, which are then computed once only by
                  finite differences.
   
              9:  NSTATE -- INTEGER                                 Input
                  On entry: if NSTATE = 1 then E04UCF is calling CONFUN
                  for the first time. This parameter setting allows the
                  user to save computation time if certain data must be
                  read or calculated only once.
   
             10:  IUSER(*) -- INTEGER array                User Workspace
   
             11:  USER(*) -- DOUBLE PRECISION array        User Workspace
                  CONFUN is called from E04UCF with the parameters IUSER
                  and USER as supplied to E04UCF. The user is free to use
                  the arrays IUSER and USER to supply information to
                  CONFUN as an alternative to using COMMON.
             CONFUN must be declared as EXTERNAL in the (sub)program
             from which E04UCF is called. Parameters denoted as
             Input must not be changed by this procedure.
   
        11:  OBJFUN -- SUBROUTINE, supplied by the user.
                                                       External Procedure
             OBJFUN must calculate the objective function F(x) and
             (optionally) the gradient g(x) for a specified n element
             vector x.
   
             Its specification is:
   
                    SUBROUTINE OBJFUN (MODE, N, X, OBJF, OBJGRD,
                   1                   NSTATE, IUSER, USER)
                    INTEGER          MODE, N, NSTATE, IUSER(*)
                    DOUBLE PRECISION X(N), OBJF, OBJGRD(N), USER(*)
   
              1:  MODE -- INTEGER                            Input/Output
                  On entry: MODE indicates the values that must be
                  assigned during each call of OBJFUN.
   
                  MODE will always have the value 2 if all components of
                  the objective gradient are specified by the user, i.e.,
                  if Derivative Level is either 1 or 3. If some gradient
                  elements are unspecified, E04UCF will call OBJFUN with
                  MODE = 0, 1 or 2.
                       If MODE = 2, compute OBJF and the available
                       components of OBJGRD.
   
                       If MODE = 1, compute all available components of
                       OBJGRD; OBJF is not required.
   
                       If MODE = 0, only OBJF needs to be computed;
                       OBJGRD is ignored.
                  On exit: MODE may be set to a negative value if the
                  user wishes to terminate the solution to the current
                  problem. If MODE is negative on exit from OBJFUN, then
                  E04UCF will terminate with IFAIL set to MODE.
   
              2:  N -- INTEGER                                      Input
                  On entry: the number, n, of variables.
   
              3:  X(N) -- DOUBLE PRECISION array                    Input
                  On entry: the vector x of variables at which the
                  objective function is to be evaluated.
   
              4:  OBJF -- DOUBLE PRECISION                         Output
                  On exit: if MODE = 0 or 2, OBJF must be set to the
                  value of the objective function at x.
   
              5:  OBJGRD(N) -- DOUBLE PRECISION array              Output
                  On exit: if MODE = 1 or 2, OBJGRD must return the
                  available components of the gradient evaluated at x.
   
              6:  NSTATE -- INTEGER                                 Input
                  On entry: if NSTATE = 1 then E04UCF is calling OBJFUN
                  for the first time. This parameter setting allows the
                  user to save computation time if certain data must be
                  read or calculated only once.
   
              7:  IUSER(*) -- INTEGER array                User Workspace
   
              8:  USER(*) -- DOUBLE PRECISION array        User Workspace
                  OBJFUN is called from E04UCF with the parameters IUSER
                  and USER as supplied to E04UCF. The user is free to use
                  the arrays IUSER and USER to supply information to
                  OBJFUN as an alternative to using COMMON.
             OBJFUN must be declared as EXTERNAL in the (sub)program
             from which E04UCF is called. Parameters denoted as
             Input must not be changed by this procedure.
   
        12:  ITER -- INTEGER                                       Output
             On exit: the number of iterations performed.
   
        13:  ISTATE(N+NCLIN+NCNLN) -- INTEGER array          Input/Output
             On entry: ISTATE need not be initialised if E04UCF is called
             with (the default) Cold Start option. The ordering of ISTATE
             is as follows. The first n elements of ISTATE refer to the
             upper and lower bounds on the variables, elements n+1
             through n+n  refer to the upper and lower bounds on A x, and
                        L                                         L
             elements n+n +1 through n+n +n  refer to the upper and lower
                         L              L  N
             bounds on c(x). When a Warm Start option is chosen, the
             elements of ISTATE corresponding to the bounds and linear
             constraints define the initial working set for the procedure
             that finds a feasible point for the linear constraints and
             bounds. The active set at the conclusion of this procedure
             and the elements of ISTATE corresponding to nonlinear
             constraints then define the initial working set for the
             first QP subproblem. Possible values for ISTATE(j) are:
   
             ISTATE(j) Meaning
   
             0         The corresponding constraint is not in the initial
                       QP working set.
   
             1         This inequality constraint should be in the
                       working set at its lower bound.
   
             2         This inequality constraint should be in the
                       working set at its upper bound.
   
             3         This equality constraint should be in the initial
                       working set. This value must not be specified
                       unless BL(j) = BU(j). The values 1,2 or 3 all have
                       the same effect when BL(j) = BU(j).
             Note that if E04UCF has been called previously with the same
             values of N, NCLIN and NCNLN, ISTATE already contains
             satisfactory values. If necessary, E04UCF will override the
             user's specification of ISTATE so that a poor choice will
             not cause the algorithm to fail. On exit: with IFAIL = 0 or
             1, the values in the array ISTATE correspond to the active
             set of the final QP subproblem, and are a prediction of the
             status of the constraints at the solution of the problem.
             Otherwise, ISTATE indicates the composition of the QP
             working set at the final iterate. The significance of each
             possible value of ISTATE(j) is as follows:
             -2        This constraint violates its lower bound by more
                       than the appropriate feasibility tolerance (see
                       the optional parameters LinearFeasibility
                       Tolerance and Nonlinear Feasibility Tolerance in
                       Section 5.1). This value can occur only when no
                       feasible point can be found for a QP subproblem.
   
             -1        This constraint violates its upper bound by more
                       than the appropriate feasibility tolerance (see
                       the optional parameters Linearear Feasibility
                       Tolerance and Nonlinear Feasibility Tolerance in
                       Section 5.1). This value can occur only when no
                       feasible point can be found for a QP subproblem.
   
             0         The constraint is satisfied to within the
                       feasibility tolerance, but is not in the working
                       set.
   
             1         This inequality constraint is included in the QP
                       working set at its upper bound.
   
             2         This inequality constraint is included in the QP
                       working set at its upper bound.
   
             3         This constraint is included in the QP working set
                       as an equality. This value of ISTATE can occur
                       only when BL(j) = BU(j).
   
        14:  C(*) -- DOUBLE PRECISION array                        Output
             Note: the dimension of the array C must be at least
             max(1,NCNLN).
             On exit: if NCNLN > 0, C(i) contains the value of the ith
             nonlinear constraint function c  at the final iterate, for
                                            i
             i=1,2,...,NCNLN. If NCNLN = 0, then the array C is not
             referenced.
   
        15:  CJAC(NROWJ,*) -- DOUBLE PRECISION array         Input/Output
             Note: the second dimension of the array CJAC must be at
             least N for NCNLN >0 and 1 otherwise On entry: in general,
             CJAC need not be initialised before the call to E04UCF.
             However, if Derivative Level = 3, the user may optionally
             set the constant elements of CJAC (see parameter NSTATE in
             the description of CONFUN). Such constant elements need not
             be re-assigned on subsequent calls to CONFUN. If NCNLN = 0,
             then the array CJAC is not referenced. On exit: if NCNLN >
             0, CJAC contains the Jacobian matrix of the nonlinear
             constraint functions at the final iterate, i.e., CJAC(i,j)
             contains the partial derivative of the ith constraint
             function with respect to the jth variable, for i=1,2,...,
             NCNLN; j = 1,2,...,N. (See the discussion of parameter CJAC
             under CONFUN.)
   
        16:  CLAMDA(N+NCLIN+NCNLN) -- DOUBLE PRECISION array Input/Output
             On entry: CLAMDA need not be initialised if E04UCF is called
             with the (default) Cold Start option. With the Warm Start
             option, CLAMDA must contain a multiplier estimate for each
             nonlinear constraint with a sign that matches the status of
             the constraint specified by the ISTATE array (as above). The
             ordering of CLAMDA is as follows; the first n elements
             contain the multipliers for the bound constraints on the
             variables, elements n+1 through n+n  contain the multipliers
                                                L
             for the general linear constraints, and elements n+n +1
                                                                 L
             through n+n +n  contain the multipliers for the nonlinear
                        L  N
             constraints. If the jth constraint is defined as 'inactive'
             by the initial value of the ISTATE array, CLAMDA(j) should
             be zero; if the jth constraint is an inequality active at
             its lower bound, CLAMDA(j) should be non-negative; if the j
             th constraint is an inequality active at its upper bound,
             CLAMDA(j) should be non-positive. On exit: the values of the
             QP multipliers from the last QP subproblem. CLAMDA(j) should
             be non-negative if ISTATE(j) = 1 and non-positive if ISTATE(
             j) = 2.
   
        17:  OBJF -- DOUBLE PRECISION                              Output
             On exit: the value of the objective function, F(x), at the
             final iterate.
   
        18:  OBJGRD(N) -- DOUBLE PRECISION array                   Output
             On exit: the gradient (or its finite-difference
             approximation) of the objective function at the final
             iterate.
   
        19:  R(NROWR,N) -- DOUBLE PRECISION array            Input/Output
             On entry: R need not be initialised if E04UCF is called with
             a Cold Start option (the default), and will be taken as the
             identity. With a Warm Start R must contain the upper-
             triangular Cholesky factor R of the initial approximation of
             the Hessian of the Lagrangian function, with the variables
             in the natural order. Elements not in the upper-triangular
             part of R are assumed to be zero and need not be assigned.
             On exit: if Hessian = No, (the default; see Section 5.1), R
                                                                 T~
             contains the upper-triangular Cholesky factor R of Q HQ, an
             estimate of the transformed and re-ordered Hessian of the
             Lagrangian at x (see (6) in Section 3). If Hessian = Yes, R
             contains the upper-triangular Cholesky factor R of H, the
             approximate (untransformed) Hessian of the Lagrangian, with
             the variables in the natural order.
   
        20:  X(N) -- DOUBLE PRECISION array                  Input/Output
             On entry: an initial estimate of the solution. On exit: the
             final estimate of the solution.
   
        21:  IWORK(LIWORK) -- INTEGER array                     Workspace
   
        22:  LIWORK -- INTEGER                                      Input
             On entry:
             the dimension of the array IWORK as declared in the
             (sub)program from which E04UCF is called.
             Constraint: LIWORK>=3*N+NCLIN+2*NCNLN.
   
        23:  WORK(LWORK) -- DOUBLE PRECISION array              Workspace
   
        24:  LWORK -- INTEGER                                       Input
             On entry:
             the dimension of the array WORK as declared in the
             (sub)program from which E04UCF is called.
             Constraints:
             if NCLIN = NCNLN = 0 then
                   LWORK >=20*N
   
             if NCNLN = 0 and NCLIN > 0 then
                              2
                   LWORK >=2*N +20*N+11*NCLIN
   
             if NCNLN > 0 and NCLIN >= 0 then
                             2
                   LWORK>=2*N +N*NCLIN+20*N*NCNLN+20*N+ 11*NCLIN+21*NCNLN
   
             If Major Print Level > 0, the required amounts of workspace
             are output on the current advisory message channel (see
             X04ABF). As an alternative to computing LIWORK and LWORK
             from the formulas given above, the user may prefer to obtain
             appropriate values from the output of a preliminary run with
             a positive value of Major Print Level and LIWORK and LWORK
             set to 1. (E04UCF will then terminate with IFAIL = 9.)
   
        25:  IUSER(*) -- INTEGER array                     User Workspace
             Note: the dimension of the array IUSER must be at least 1.
             IUSER is not used by E04UCF, but is passed directly to
             routines CONFUN and OBJFUN and may be used to pass
             information to those routines.
   
        26:  USER(*) -- DOUBLE PRECISION array             User Workspace
             Note: the dimension of the array USER must be at least 1.
             USER is not used by E04UCF, but is passed directly to
             routines CONFUN and OBJFUN and may be used to pass
             information to those routines.
   
        27:  IFAIL -- INTEGER                                Input/Output
             On entry: IFAIL must be set to 0, -1 or 1. Users who are
             unfamiliar with this parameter should refer to the Essential
             Introduction for details.
   
             On exit: IFAIL = 0 unless the routine detects an error or
             gives a warning (see Section 6).
   
             For this routine, because the values of output parameters
             may be useful even if IFAIL /=0 on exit, users are
             recommended to set IFAIL to -1 before entry. It is then
             essential to test the value of IFAIL on exit.
   
             E04UCF returns with IFAIL = 0 if the iterates have
             converged to a point x that satisfies the first-order Kuhn-
             Tucker conditions to the accuracy requested by the optional
             parameter Optimality Tolerance (see Section 5.1), i.e., the
             projected gradient and active constraint residuals are
             negligible at x.
   
             The user should check whether the following four conditions
             are satisfied:
             (i)   the final value of Norm Gz is significantly less than
                   that at the starting point;
   
             (ii)  during the final major iterations, the values of Step
                   and ItQP are both one;
   
             (iii) the last few values of both Norm Gz and Norm C become
                   small at a fast linear rate;
   
             (iv)  Cond Hz is small.
             If all these conditions hold, x is almost certainly a local
             minimum of (1). (See Section 9 for a specific example.)
   
        5.1. Optional Input Parameters
   
        Several optional parameters in E04UCF define choices in the
        behaviour of the routine. In order to reduce the number of formal
        parameters of E04UCF these optional parameters have associated
        default values (see Section 5.1.3) that are appropriate for most
        problems. Therefore the user need only specify those optional
        parameters whose values are to be different from their default
        values.
   
        The remainder of this section can be skipped by users who wish to
        use the default values for all optional prameters. A complete
        list of optional parameters and their default values is given in
        Section 5.1.3
   
        5.1.1. Specification of the optional parameters
   
        Optional parameters may be specified by calling one, or both, of
        E04UDF and E04UEF prior to a call to E04UCF.
   
        E04UDF reads options from an external options file, with Begin
        and End as the first and last lines respectively and each
        intermediate line defining a single optional parameter. For
        example,
   
              Begin
                Print Level = 1
              End
   
        The call
   
              CALL E04UDF (IOPTNS, INFORM)
   
        can then be used to read the file on unit IOPTNS. INFORM will be
        zero on successful exit. E04UDF should be consulted for a full
        description of this method of supplying optional parameters.
   
        E04UEF can be called directly to supply options, one call being
        necessary for each optional parameter. For example,
   
              CALL E04UEF (`Print level = 1')
   
        E04UEF should be consulted for a full description of this method
        of supplying optional parameters.
   
        All optional parameters not specified by the user are set to
        their default values. Optional parameters specified by the user
        are unaltered by E04UCF (unless they define invalid values) and
        so remain in effect for subsequent calls to E04UCF, unless
        altered by the user.
   
        5.1.2. Description of the optional parameters
   
        The following list (in alphabetical order) gives the valid
        options. For each option, we give the keyword, any essential
        optional qualifiers, the default value, and the definition. The
        minimum valid abbreviation of each keyword is underlined. If no
        characters of an optional qualifier are underlined, the qualifier
        may be omitted. The letter a denotes a phrase (character string)
        that qualifies an option. The letters i and r denote INTEGER and
        DOUBLE PRECISION values required with certain options. The number
        (epsilon) is a generic notation for machine precision (see
        X02AJF(*) ), and (epsilon)  denotes the relative precision of the
                                  R
        objective function (the optional parameter Function Precision see
        below).
   
        Central Difference Interval r Default values are computed
   
        If the algorithm switches to central differences because the
        forward-difference approximation is not sufficiently accurate,
        the value of r is used as the difference interval for every
        component of x. The use of finite-differences is discussed
        further below under the optional parameter Difference Interval.
   
        Cold Start Default = Cold Start
   
        Warm Start
   
           (AXIOM parameter STA, warm start when .TRUE.)
   
        This option controls the specification of the initial working set
        in both the procedure for finding a feasible point for the linear
        constraints and bounds, and in the first QP subproblem
        thereafter. With a Cold Start, the first working set is chosen by
        E04UCF based on the values of the variables and constraints at
        the initial point. Broadly speaking, the initial working set will
        include equality constraints and bounds or inequality constraints
        that violate or 'nearly' satisfy their bounds (within Crash
        Tolerance; see below). With a Warm Start, the user must set the
        ISTATE array and define CLAMDA and R as discussed in Section 5.
        ISTATE values associated with bounds and linear constraints
        determine the initial working set of the procedure to find a
        feasible point with respect to the bounds and linear constraints.
        ISTATE values associated with nonlinear constraints determine the
        initial working set of the first QP subproblem after such a
        feasible point has been found. E04UCF will override the user's
        specification of ISTATE if necessary, so that a poor choice of
        the working set will not cause a fatal error. A warm start will
        be advantageous if a good estimate of the initial working set is
        available - for example, when E04UCF is called repeatedly to
        solve related problems.
   
        Crash Tolerance r Default = 0.01
   
           (AXIOM parameter CRA)
   
        This value is used in conjunction with the optional parameter
        Cold Start (the default value). When making a cold start, the QP
        algorithm in E04UCF must select an initial working set. When r>=0
        , the initial working set will include (if possible) bounds or
        general inequality constraints that lie within r of their bounds.
                                                 T
        In particular, a constraint of the form a x>=l will be included
                                                 j
                                        T
        in the initial working set if |a x-l|<=r(1+|l|). If r<0 or r>1,
                                        j
        the default value is used.
   
        Defaults
   
        This special keyword may be used to reset the default values
        following a call to E04UCF.
   
        Derivative Level i Default = 3
   
           (AXIOM parameter DER)
   
        This parameter indicates which derivatives are provided by the
        user in subroutines OBJFUN and CONFUN. The possible choices for i
        are the following.
   
            i    Meaning
   
            3    All objective and constraint gradients are provided by
                 the user.
   
            2    All of the Jacobian is provided, but some components of
                 the objective gradient are not specified by the user.
   
            1    All elements of the objective gradient are known, but
                 some elements of the Jacobian matrix are not specified
                 by the user.
   
            0    Some elements of both the objective gradient and the
                 Jacobian matrix are not specified by the user.
   
        The value i=3 should be used whenever possible, since E04UCF is
        more reliable and will usually be more efficient when all
        derivatives are exact.
   
        If i=0 or 2, E04UCF will estimate the unspecified components of
        the objective gradient, using finite differences. The computation
        of finite-difference approximations usually increases the total
        run-time, since a call to OBJFUN is required for each unspecified
        element. Furthermore, less accuracy can be attained in the
        solution (see Chapter 8 of Gill et al [10], for a discussion of
        limiting accuracy).
   
        If i=0 or 1, E04UCF will approximate unspecified elements of the
        Jacobian. One call to CONFUN is needed for each variable for
        which partial derivatives are not available. For example, if the
        Jacobian has the form
   
                                    (* * * *)
                                    (* ? ? *)
                                    (* * ? *)
                                    (* * * *)
   
        where '*' indicates an element provided by the user and '?'
        indicates an unspecified element, E04UCF will call CONFUN twice:
        once to estimate the missing element in column 2, and again to
        estimate the two missing elements in column 3. (Since columns 1
        and 4 are known, they require no calls to CONFUN.)
   
        At times, central differences are used rather than forward
        differences, in which case twice as many calls to OBJFUN and
        CONFUN are needed. (The switch to central differences is not
        under the user's control.)
   
        Difference Interval r Default values are computed
   
           (AXIOM parameter DIF)
   
        This option defines an interval used to estimate gradients by
        finite differences in the following circumstances:
   
        (a)   For verifying the objective and/or constraint gradients
              (see the description of Verify, below).
   
        (b)   For estimating unspecified elements of the objective
              gradient of the Jacobian matrix.
   
        In general, a derivative with respect to the jth variable is
                                                                      ^
        approximated using the interval (delta) , where (delta) =r(1+|x |)
                                               j               j       j
                ^
        with x the first point feasible with respect to the bounds and
        linear constraints. If the functions are well scaled, the
        resulting derivative approximation should be accurate to O(r).
   
        See Gill et al [10] for a discussion of the accuracy in finite-
        difference approximations.
   
        If a difference interval is not specified by the user, a finite-
        difference interval will be computed automatically for each
        variable by a procedure that requires up to six calls of CONFUN
        and OBJFUN for each component. This option is recommended if the
        function is badly scaled or the user wishes to have E04UCF
        determine constant elements in the objective and constraint
        gradients (see the descriptions of CONFUN and OBJFUN in
        Section 5).
   
                                            _________
        Feasibility Tolerance r Default = \/(epsilon)
   
           (AXIOM parameter FEA)
   
        The scalar r defines the maximum acceptable absolute violations
        in linear and nonlinear constraints at a 'feasible' point; i.e.,
        a constraint is considered satisfied if its violation does not
        exceed r. If r<(epsilon) or r>=1, the default value is used.
        Using this keyword sets both optional parameters Linear
        Feasibility Tolerance and Nonlinear Feasibility Tolerance to r,
        if (epsilon)<=r<1. (Additional details are given below under the
        descriptions of these parameters.)
   
                                                0.9
        Function Precision r Default = (epsilon)
   
           (AXIOM parameter FUN)
   
        This parameter defines (epsilon) , which is intended to be a
                                        R
        measure of the accuracy with which the problem functions f and c
        can be computed. If r<(epsilon) or r>=1, the default value is
        used. The value of (epsilon)  should reflect the relative
                                    R
        precision of 1+|F(x)|; i.e., (epsilon)  acts as a relative
                                              R
        precision when |F| is large, and as an absolute precision when
        |F| is small. For example, if F(x) is typically of order 1000 and
        the first six significant digits are known to be correct, an
        appropriate value for (epsilon)  would be 1.0E-6. In contrast, if
                                       R
                                     -4
        F(x) is typically of order 10   and the first six significant
        digits are known to be correct, an appropriate value for
        (epsilon)  would be 1.0E-10. The choice of (epsilon)  can be
                 R                                          R
        quite complicated for badly scaled problems; see Chapter 8 of
        Gill et al [10] for a discussion of scaling techniques. The
        default value is appropriate for most simple functions that are
        computed with full accuracy. However, when the accuracy of the
        computed function values is known to be significantly worse than
        full precision, the value of (epsilon)  should be large enough so
                                              R
        that E04UCF will not attempt to distinguish between function
        values that differ by less than the error inherent in the
        calculation.
   
        Hessian No Default = No
   
        Hessian Yes
   
           (No AXIOM parameter - fixed as Yes)
   
        This option controls the contents of the upper-triangular matrix
        R (see Section 5). E04UCF works exclusively with the transformed
        and re-ordered Hessian H  (6), and hence extra computation is
                                Q
        required to form the Hessian itself. If Hessian = No, R contains
        the Cholesky factor of the transformed and re-ordered Hessian. If
        Hessian = Yes the Cholesky factor of the approximate Hessian
        itself is formed and stored in R. The user should select Hessian
        = Yes if a warm start will be used for the next call to E04UCF.
   
                                          10
        Infinite Bound Size r Default = 10
   
           (AXIOM parameter INFB)
   
        If r>0, r defines the 'infinite' bound BIGBND in the definition
        of the problem constraints. Any upper bound greater than or equal
        to BIGBND will be regarded as plus infinity (and similarly for a
        lower bound less than or equal to -BIGBND). If r<=0, the default
        value is used.
   
                                                    10
        Infinite Step Size r Default = max(BIGBND,10  )
   
           (AXIOM parameter INFS)
   
        If r>0, r specifies the magnitude of the change in variables that
        is treated as a step to an unbounded solution. If the change in x
        during an iteration would exceed the value of Infinite Step Size,
        the objective function is considered to be unbounded below in the
        feasible region. If r<=0, the default value is used.
   
        Iteration limit i Default = max(50,3(n+n )+10n )
                                                L     N
   
        See Major Iteration Limit below.
   
                                                    _________
        Linear Feasibility Tolerance r  Default = \/(epsilon)
                                      1
   
           (AXIOM parameter LINF)
   
                                                       _________
        Nonlinear Feasibility Tolerance r  Default = \/(epsilon) if
                                         2
   
           (AXIOM parameter NONF)
                                           0.33
        Derivative Level >= 2 and (epsilon)     otherwise
   
        The scalars r  and r  define the maximum acceptable absolute
                     1      2
        violations in linear and nonlinear constraints at a 'feasible'
        point; i.e., a linear constraint is considered satisfied if its
        violation does not exceed r , and similarly for a nonlinear
                                   1
        constraint and r . If r <(epsilon) or r >=1, the default value is
                        2      i               i
        used, for i=1,2.
   
        On entry to E04UCF, an iterative procedure is executed in order
        to find a point that satisfies the linear constraint and bounds
        on the variables to within the tolerance r . All subsequent
                                                  1
        iterates will satisfy the linear constraints to within the same
        tolerance (unless r  is comparable to the finite-difference
                           1
        interval).
   
        For nonlinear constraints, the feasibility tolerance r  defines
                                                              2
        the largest constraint violation that is acceptable at an optimal
        point. Since nonlinear constraints are generally not satisfied
        until the final iterate, the value of Nonlinear Feasibility
        Tolerance acts as a partial termination criterion for the
        iterative sequence generated by E04UCF (see the discussion of
        Optimality Tolerance).
   
        These tolerances should reflect the precision of the
        corresponding constraints. For example, if the variables and the
        coefficients in the linear constraints are of order unity, and
        the latter are correct to about 6 decimal digits, it would be
                                       - 6
        appropriate to specify r  as 10   .
                                1
   
        Linesearch Tolerance r Default = 0.9
   
           (AXIOM parameter LINT)
   
   
        The value r (0 <= r < 1) controls the accuracy with which the
        step (alpha) taken during each iteration approximates a minimum
        of the merit function along the search direction (the smaller the
        value of r, the more accurate the linesearch). The default value
        r=0.9 requests an inaccurate search, and is appropriate for most
        problems, particularly those with any nonlinear constraints.
   
        If there are no nonlinear constraints, a more accurate search may
        be appropriate when it is desirable to reduce the number of major
        iterations - for example, if the objective function is cheap to
        evaluate, or if a substantial number of gradients are
        unspecified.
   
        List Default = List
   
        Nolist
   
           (AXIOM parameter LIST)
   
        Normally each optional parameter specification is printed as it
        is supplied. Nolist may be used to suppress the printing and List
        may be used to restore printing.
   
        Major Iteration Limit i Default = max(50,3(n+n )+10n )
                                                      L     N
   
        Iteration Limit
   
        Iters
   
        Itns
   
           (AXIOM parameter MAJI)
   
        The value of i specifies the maximum number of major iterations
        allowed before termination. Setting i=0 and Major Print Level> 0
        means that the workspace needed will be computed and printed, but
        no iterations will be performed.
   
        Major Print level i Default = 10
   
        Print Level
   
           (AXIOM parameter MAJP)
   
        The value of i controls the amount of printout produced by the
        major iterations of E04UCF. (See also Minor Print level below.)
        The levels of printing are indicated below.
   
        i        Output
   
        0        No output.
   
        1        The final solution only.
   
        5        One line for each major iteration (no printout of the
                 final solution).
   
        >=10     The final solution and one line of output for each
                 iteration.
   
        >=20     At each major iteration, the objective function, the
                 Euclidean norm of the nonlinear constraint violations,
                 the values of the nonlinear constraints (the vector c),
                 the values of the linear constraints (the vector A x),
                                                                   L
                 and the current values of the variables (the vector x).
   
        >=30     At each major iteration, the diagonal elements of the
                 matrix T associated with the TQ factorization (5) of the
                 QP working set, and the diagonal elements of R, the
                 triangular factor of the transformed and re-ordered
                 Hessian (6).
   
        Minor Iteration Limit i Default = max(50,3(n+n +n ))
                                                      L  N
   
           (AXIOM parameter MINI)
   
        The value of i specifies the maximum number of iterations for the
        optimality phase of each QP subproblem.
   
        Minor Print Level i Default = 0
   
           (AXIOM parameter MINP)
   
        The value of i controls the amount of printout produced by the
        minor iterations of E04UCF, i.e., the iterations of the quadratic
        programming algorithm. (See also Major Print Level, above.) The
        following levels of printing are available.
   
        i        Output
   
        0        No output.
   
        1        The final QP solution.
   
        5        One line of output for each minor iteration (no printout
                 of the final QP solution).
   
        >=10     The final QP solution and one brief line of output for
                 each minor iteration.
   
        >=20     At each minor iteration, the current estimates of the QP
                 multipliers, the current estimate of the QP search
                 direction, the QP constraint values, and the status of
                 each QP constraint.
   
        >=30     At each minor iteration, the diagonal elements of the
                 matrix T associated with the TQ factorization (5) of the
                 QP working set, and the diagonal elements of the
                 Cholesky factor R of the transformed Hessian (6).
   
                                                      _________
        Nonlinear Feasibility Tolerance r Default = \/(epsilon)
   
        See Linear Feasibility Tolerance, above.
   
                                                  0.8
        Optimality Tolerance r Default = (epsilon)
   
           (AXIOM parameter OPT)
   
        The parameter r ((epsilon) <=r<1) specifies the accuracy to which
                                  R
        the user wishes the final iterate to approximate a solution of
        the problem. Broadly speaking, r indicates the number of correct
        figures desired in the objective function at the solution. For
                           - 6
        example, if r is 10    and E04UCF terminates successfully, the
        final value of F should have approximately six correct figures.
        If r<(epsilon)  or r>=1 the default value is used.
                      R
   
        E04UCF will terminate successfully if the iterative sequence of x
        -values is judged to have converged and the final point satisfies
        the first-order Kuhn-Tucker conditions (see Section 3). The
        sequence of iterates is considered to have converged at x if
   
                                        _
                       (alpha) ||p||<=\/r(1+||x||),                  (8a)
   
        where p is the search direction and (alpha) the step length from
        (3). An iterate is considered to satisfy the first-order
        conditions for a minimum if
   
                     T         _
                  ||Z g  ||<=\/r(1+max(1+|F(x)|,||g  ||))            (8b)
                       FR                          FR
   
        and
   
                          |res |<=ftol for all j,                    (8c)
                              j
   
               T
        where Z g   is the projected gradient (see Section 3), g   is the
                 FR                                             FR
        gradient of F(x) with respect to the free variables, res  is the
                                                                j
        violation of the jth active nonlinear constraint, and ftol is the
        Nonlinear Feasibility Tolerance.
   
        Step Limit r Default = 2.0
   
           (AXIOM parameter STE)
   
        If r>0, r specifies the maximum change in variables at the first
                                                              bx
        step of the linesearch. In some cases, such as F(x)=ae   or
               b
        F(x)=ax , even a moderate change in the components of x can lead
        to floating-point overflow. The parameter r is therefore used to
        encourage evaluation of the problem functions at meaningful
                                                           ~
        points. Given any major iterate x, the first point x at which F
        and c are evaluated during the linesearch is restricted so that
   
                               ~
                             ||x-x|| <=r(1+||x|| ).
                                    2           2
   
        The linesearch may go on and evaluate F and c at points further
        from x if this will result in a lower value of the merit
        function. In this case, the character L is printed at the end of
        the optional line of printed output, (see Section 5.2). If L is
        printed for most of the iterations, r should be set to a larger
        value.
   
        Wherever possible, upper and lower bounds on x should be used to
        prevent evaluation of nonlinear functions at wild values. The
        default value Step Limit = 2.0 should not affect progress on
        well-behaved functions, but values 0.1 or 0.01 may be helpful
        when rapidly varying functions are present. If a small value of
        Step Limit is selected, a good starting point may be required. An
        important application is to the class of nonlinear least-squares
        problems. If r<=0, the default value is used.
   
        Start Objective Check At Variable k Default = 1
   
           (AXIOM parameter STAO)
   
        Start Constraint Check At Variable k Default = 1
   
           (AXIOM parameter STAC)
   
        Stop Objective Check At Variable l Default = n
   
           (AXIOM parameter STOO)
   
        Stop Constraint Check At Variable l Default = n
   
           (AXIOM parameter STOC)
   
        These keywords take effect only if Verify Level > 0 (see below).
        They may be used to control the verification of gradient elements
        computed by subroutines OBJFUN and CONFUN. For example, if the
        first 30 components of the objective gradient appeared to be
        correct in an earlier run, so that only component 31 remains
        questionable, it is reasonable to specify Start Objective Check
        At Variable 31. If the first 30 variables appear linearly in the
        objective, so that the corresponding gradient elements are
        constant, the above choice would also be appropriate.
   
        Verify Level i Default = 0
   
        Verify No
   
        Verify Level - 1
   
        Verify Level 0
   
        Verify Objective Gradients
   
        Verify Level 1
   
        Verify Constraint Gradients
   
        Verify Level 2
   
        Verify
   
        Verify Yes
   
        Verify Gradients
   
        Verify Level 3
   
           (AXIOM parameter VE)
   
        These keywords refer to finite-difference checks on the gradient
        elements computed by the user-provided subroutines OBJFUN and
        CONFUN. (Unspecified gradient components are not checked.) It is
        possible to specify Verify Levels 0-3 in several ways, as
        indicated above. For example, the nonlinear objective gradient
        (if any) will be verified if either Verify Objective Gradients or
        Verify Level 1 is specified. Similarly, the objective and the
        constraint gradients will be verified if Verify Yes or Verify
        Level 3 or Verify is specified.
   
        If 0<=i<=3, gradients will be verified at the first point that
        satisfies the linear constraints and bounds. If i=0, only a '
        cheap' test will be performed, requiring one call to OBJFUN and
        one call to CONFUN. If 1<=i<=3, a more reliable (but more
        expensive) check will be made on individual gradient components,
        within the ranges specified by the Start and Stop keywords
        described above. A result of the form OK or BAD? is printed by
        E04UCF to indicate whether or not each component appears to be
        correct.
   
        If 10<=i<=13, the action is the same as for i - 10, except that
        it will take place at the user-specified initial value of x.
   
        We suggest that Verify Level 3 be specified whenever a new
        function routine is being developed.
   
        5.1.3. Optional parameter checklist and default values
   
        For easy reference, the following list shows all the valid
        keywords and their default values. The symbol (epsilon)
        represents the machine precision (see X02AJF(*) ).
   
        Optional Parameters      Default Values
   
   
   
        Central difference       Computed automatically
        interval
   
        Cold/Warm start          Cold start
   
        Crash tolerance          0.01
   
        Defaults
   
        Derivative level         3
   
        Difference interval      Computed automatically
   
                                   _________
        Feasibility tolerance    \/(epsilon)
   
                                          0.9
        Function precision       (epsilon)
   
        Hessian                  No
   
                                   10
        Infinite bound size      10
   
                                   10
        Infinite step size       10
   
                                   _________
        Linear feasibility       \/(epsilon)
        tolerance
   
        Linesearch tolerance     0.9
   
        List/Nolist              List
   
        Major iteration limit    max(50,3(n+n )+10n )
                                             L     N
   
        Major print level        10
   
        Minor iteration limit    max(50,3(n+n +n ))
                                             L  N
   
        Minor print level        0
   
                                   _________
        Nonlinear feasibility    \/(epsilon) if Derivative Level >= 2
        tolerance                                   0.33
                                 otherwise (epsilon)
   
                                          0.8
        Optimality tolerance     (epsilon)
                                          R
   
        Step limit               2.0
   
        Start objective check    1
   
        Start constraint check   1
   
        Stop objective check     n
   
        Stop constraint check    n
   
        Verify level             0
   
        5.2. Description of Printed Output
   
        The level of printed output from E04UCF is controlled by the user
        (see the description of Major Print Level and Minor Print Level
        in Section 5.1). If Minor Print Level > 0, output is obtained
        from the subroutines that solve the QP subproblem. For a detailed
        description of this information the reader should refer to
        E04NCF(*).
   
        When Major Print Level >= 5, the following line of output is
        produced at every major iteration of E04UCF. In all cases, the
        values of the quantities printed are those in effect on
        completion of the given iteration.
   
        Itn            is the iteration count.
   
        ItQP           is the sum of the iterations required by the
                       feasibility and optimality phases of the QP
                       subproblem. Generally, ItQP will be 1 in the later
                       iterations, since theoretical analysis predicts
                       that the correct active set will be identified
                       near the solution (see Section 3).
   
                       Note that ItQP may be greater than the Minor
                       Iteration Limit if some iterations are required
                       for the feasibility phase.
   
        Step           is the step (alpha) taken along the computed
                       search direction. On reasonably well-behaved
                       problems, the unit step will be taken as the
                       solution is approached.
   
        Nfun           is the cumulative number of evaluations of the
                       objective function needed for the linesearch.
                       Evaluations needed for the estimation of the
                       gradients by finite differences are not included.
                       Nfun is printed as a guide to the amount of work
                       required for the linesearch.
   
        Merit          is the value of the augmented Lagrangian merit
                       function (12) at the current iterate. This
                       function will decrease at each iteration unless it
                       was necessary to increase the penalty parameters
                       (see Section 8.2). As the solution is approached,
                       Merit will converge to the value of the objective
                       function at the solution.
   
                       If the QP subproblem does not have a feasible
                       point (signified by I at the end of the current
                       output line), the merit function is a large
                       multiple of the constraint violations, weighted by
                       the penalty parameters. During a sequence of major
                       iterations with infeasible subproblems, the
                       sequence of Merit values will decrease
                       monotonically until either a feasible subproblem
                       is obtained or E04UCF terminates with IFAIL = 3
                       (no feasible point could be found for the
                       nonlinear constraints).
   
                       If no nonlinear constraints are present (i.e.,
                       NCNLN = 0), this entry contains Objective, the
                       value of the objective function F(x). The
                       objective function will decrease monotonically to
                       its optimal value when there are no nonlinear
                       constraints.
   
        Bnd            is the number of simple bound constraints in the
                       predicted active set.
   
        Lin            is the number of general linear constraints in the
                       predicted active set.
   
        Nln            is the number of nonlinear constraints in the
                       predicted active set (not printed if NCNLN is
                       zero).
   
        Nz             is the number of columns of Z (see Section 8.1).
                       The value of Nz is the number of variables minus
                       the number of constraints in the predicted active
                       set; i.e., Nz = n-(Bnd + Lin + Nln).
   
        Norm Gf        is the Euclidean norm of g  , the gradient of the
                                                 FR
                       objective function with respect to the free
                       variables, i.e.,variables not currently held at a
                       bound.
   
                             T
        Norm Gz        is ||Z g  ||, the Euclidean norm of the projected
                               FR
                       gradient (see Section 8.1). Norm  Gz will be
                       approximately zero in the neighbourhood of a
                       solution.
   
        Cond H         is a lower bound on the condition number of the
                       Hessian approximation H.
   
        Cond Hz        is a lower bound on the condition number of the
                       projected Hessian approximation H  (
                                                        z
                            T      T
                       (H =Z H  Z=R R ; see (6) and (12) in Sections 3
                         z    FR   z z
                       and 8.1). The larger this number, the more
                       difficult the problem.
   
        Cond T         is a lower bound on the condition number of the
                       matrix of predicted active constraints.
   
        Norm C         is the Euclidean norm of the residuals of
                       constraints that are violated or in the predicted
                       active set (not printed if NCNLN is zero). Norm C
                       will be approximately zero in the neighbourhood of
                       a solution.
   
        Penalty        is the Euclidean norm of the vector of penalty
                       parameters used in the augumented Lagrangian merit
                       function (not printed if NCNLN is zero).
   
        Conv           is a three-letter indication of the status of the
                       three convergence tests (8a)-(8c) defined in the
                       description of the optional parameter Optimality
                       Tolerance in Section 5.1 Each letter is T if the
                       test is satisfied, and F otherwise. The three
                       tests indicate whether:
                       (a)   the sequence of iterates has converged;
   
                       (b)   the projected gradient (Norm  Gz) is
                             sufficiently small; and
   
                       (c)   the norm of the residuals of constraints in
                             the predicted active set (Norm  C) is small
                             enough.
                       If any of these indicators is F when E04UCF
                       terminates with IFAIL = 0, the user should check
                       the solution carefully.
   
        M              is printed if the Quasi-Newton update was modified
                       to ensure that the Hessian approximation is
                       positive-definite (see Section 8.3).
   
        I              is printed if the QP subproblem has no feasible
                       point.
   
        C              is printed if central differences were used to
                       compute the unspecified objective and constraint
                       gradients. If the value of Step is zero, the
                       switch to central differences was made because no
                       lower point could be found in the linesearch. (In
                       this case, the QP subproblem is resolved with the
                       central-difference gradient and Jacobian.) If the
                       value of Step is non-zero, central differences
                       were computed because Norm  Gz and Norm  C imply
                       that x is close to a Kuhn-Tucker point.
   
        L              is printed if the linesearch has produced a
                       relative change in x greater than the value
                       defined by the optional parameter Step Limit. If
                       this output occurs frequently during later
                       iterations of the run, Step Limit should be set to
                       a larger value.
   
        R              is printed if the approximate Hessian has been
                       refactorized. If the diagonal condition estimator
                       of R indicates that the approximate Hessian is
                       badly conditioned, the approximate Hessian is
                       refactorized using column interchanges. If
                       necessary, R is modified so that its diagonal
                       condition estimator is bounded.
   
        When Major Print Level = 1 or Major Print Level >= 10, the
        summary printout at the end of execution of E04UCF includes a
        listing of the status of every variable and constraint. Note that
        default names are assigned to all variables and constraints.
   
        The following describes the printout for each variable.
   
        Varbl          gives the name (V) and index j=1,2,...,n of the
                       variable.
   
        State          gives the state of the variable in the predicted
                       active set (FR if neither bound is in the active
                       set, EQ if a fixed variable, LL if on its lower
                       bound, UL if on its upper bound). If the variable
                       is predicted to lie outside its upper or lower
                       bound by more than the feasibility tolerance,
                       State will be ++ or -- respectively. (The latter
                       situation can occur only when there is no feasible
                       point for the bounds and linear constraints.)
   
        Value          is the value of the variable at the final
                       iteration.
   
        Lower bound    is the lower bound specified for the variable.
                       (None indicates that BL(j)<=- BIGBND.)
   
        Upper bound    is the upper bound specified for the variable.
                       (None indicates that BL(j)>=BIGBND.)
   
        Lagr Mult      is the value of the Lagrange-multiplier for the
                       associated bound constraint. This will be zero if
                       State is FR. If x is optimal, the multiplier
                       should be non-negative if State is LL, and non-
                       positive if State is UL.
   
        Residual       is the difference between the variable Value and
                       the nearer of its bounds BL(j) and BU(j).
   
        The printout for general constraints is the same as for
        variables, except for the following:
   
        L Con      is the name (L) and index i, for i = 1,2,...,NCLIN of
                   a linear constraint.
   
        N Con      is the name (N) and index i, for i = 1,2,...,NCNLN of
                   a nonlinear constraint.
   
        6. Error Indicators and Warnings
   
        Errors or warnings specified by the routine:
   
        If on entry IFAIL = 0 or -1, explanatory error messages are
        output on the current error message unit (as defined by X04AAF).
   
        The input data for E04UCF should always be checked (even if
        E04UCF terminates with IFAIL=0).
   
        Note that when Print Level>0, a short description of IFAIL is
        printed.
   
        Errors and diagnostics indicated by IFAIL, together with some
        recommendations for recovery are indicated below.
   
        IFAIL= 1
             The final iterate x satisfies the first-order Kuhn-Tucker
             conditions to the accuracy requested, but the sequence of
             iterates has not yet converged. E04UCF was terminated
             because no further improvement could be made in the merit
             function.
   
             This value of IFAIL may occur in several circumstances. The
             most common situation is that the user asks for a solution
             with accuracy that is not attainable with the given
             precision of the problem (as specified by Function Precision
             see Section 5). This condition will also occur if, by
             chance, an iterate is an 'exact' Kuhn-Tucker point, but the
             change in the variables was significant at the previous
             iteration. (This situation often happens when minimizing
             very simple functions, such as quadratics.)
   
             If the four conditions listed in Section 5 for IFAIL = 0 are
             satisfied, x is likely to be a solution of (1) even if IFAIL
             = 1.
   
        IFAIL= 2
             E04UCF has terminated without finding a feasible point for
             the linear constraints and bounds, which means that no
             feasible point exists for the given value of Linear
             Feasibility Tolerance (see Section 5.1). The user should
             check that there are no constraint redundancies. If the data
             for the constraints are accurate only to an absolute
             precision (sigma), the user should ensure that the value of
             the optional parameter Linear Feasibility Tolerance is
             greater than (sigma). For example, if all elements of A are
             of order unity and are accurate to only three decimal
                                                                       -3
             places, Linear Feasibility Tolerance should be at least 10  .
   
        IFAIL= 3
             No feasible point could be found for the nonlinear
             constraints. The problem may have no feasible solution. This
             means that there has been a sequence of QP subproblems for
             which no feasible point could be found (indicated by I at
             the end of each terse line of output). This behaviour will
             occur if there is no feasible point for the nonlinear
             constraints. (However, there is no general test that can
             determine whether a feasible point exists for a set of
             nonlinear constraints.) If the infeasible subproblems occur
             from the very first major iteration, it is highly likely
             that no feasible point exists. If infeasibilities occur when
             earlier subproblems have been feasible, small constraint
             inconsistencies may be present. The user should check the
             validity of constraints with negative values of ISTATE. If
             the user is convinced that a feasible point does exist,
             E04UCF should be restarted at a different starting point.
   
        IFAIL= 4
             The limiting number of iterations (determined by the
             optional parameter Major Iteration Limit see Section 5.1)
             has been reached.
   
             If the algorithm appears to be making progress, Major
             Iteration Limit may be too small. If so, increase its value
             and rerun E04UCF (possibly using the Warm Start option). If
             the algorithm seems to be 'bogged down', the user should
             check for incorrect gradients or ill-conditioning as
             described below under IFAIL = 6.
   
             Note that ill-conditioning in the working set is sometimes
             resolved automatically by the algorithm, in which case
             performing additional iterations may be helpful. However,
             ill-conditioning in the Hessian approximation tends to
             persist once it has begun, so that allowing additional
             iterations without altering R is usually inadvisable. If the
             quasi-Newton update of the Hessian approximation was
             modified during the latter iterations (i.e., an M occurs at
             the end of each terse line), it may be worthwhile to try a
             warm start at the final point as suggested above.
   
        IFAIL= 6
             x does not satisfy the first-order Kuhn-Tucker conditions,
             and no improved point for the merit function could be found
             during the final line search.
   
             A sufficient decrease in the merit function could not be
             attained during the final line search. This sometimes occurs
             because an overly stringent accuracy has been requested,
             i.e., Optimality Tolerance is too small. In this case the
             user should apply the four tests described under IFAIL = 0
             above to determine whether or not the final solution is
             acceptable (see Gill et al [10], for a discussion of the
             attainable accuracy).
   
             If many iterations have occurred in which essentially no
             progress has been made and E04UCF has failed completely to
             move from the initial point then subroutines OBJFUN or
             CONFUN may be incorrect. The user should refer to comments
             below under IFAIL = 7 and check the gradients using the
             Verify parameter. Unfortunately, there may be small errors
             in the objective and constraint gradients that cannot be
             detected by the verification process. Finite-difference
             approximations to first derivatives are catastrophically
             affected by even small inaccuracies. An indication of this
             situation is a dramatic alteration in the iterates if the
             finite-difference interval is altered. One might also
             suspect this type of error if a switch is made to central
             differences even when Norm Gz and Norm C are large.
   
             Another possibility is that the search direction has become
             inaccurate because of ill-conditioning in the Hessian
             approximation or the matrix of constraints in the working
             set; either form of ill-conditioning tends to be reflected
             in large values of ItQP (the number of iterations required
             to solve each QP subproblem).
   
             If the condition estimate of the projected Hessian (Cond Hz)
             is extremely large, it may be worthwhile to rerun E04UCF
             from the final point with the Warm Start option. In this
             situation, ISTATE should be left unaltered and R should be
             reset to the identity matrix.
   
             If the matrix of constraints in the working set is ill-
             conditioned (i.e., Cond T is extremely large), it may be
             helpful to run E04UCF with a relaxed value of the
             Feasibility Tolerance (Constraint dependencies are often
             indicated by wide variations in size in the diagonal
             elements of the matrix T, whose diagonals will be printed
             for Major Print Level >= 30).
   
        IFAIL= 7
             The user-provided derivatives of the objective function
             and/or nonlinear constraints appear to be incorrect.
   
             Large errors were found in the derivatives of the objective
             function and/or nonlinear constraints. This value of IFAIL
             will occur if the verification process indicated that at
             least one gradient or Jacobian component had no correct
             figures. The user should refer to the printed output to
             determine which elements are suspected to be in error.
   
             As a first-step, the user should check that the code for the
             objective and constraint values is correct - for example, by
             computing the function at a point where the correct value is
             known. However, care should be taken that the chosen point
             fully tests the evaluation of the function. It is remarkable
             how often the values x=0 or x=1 are used to test function
             evaluation procedures, and how often the special properties
             of these numbers make the test meaningless.
   
             Special care should be used in this test if computation of
             the objective function involves subsidiary data communicated
             in COMMON storage. Although the first evaluation of the
             function may be correct, subsequent calculations may be in
             error because some of the subsidiary data has accidently
             been overwritten.
   
             Errors in programming the function may be quite subtle in
             that the function value is 'almost' correct. For example,
             the function may not be accurate to full precision because
             of the inaccurate calculation of a subsidiary quantity, or
             the limited accuracy of data upon which the function
             depends. A common error on machines where numerical
             calculations are usually performed in double precision is to
             include even one single-precision constant in the
             calculation of the function; since some compilers do not
             convert such constants to double precision, half the correct
             figures may be lost by such a seemingly trivial error.
   
        IFAIL= 9
             An input parameter is invalid. The user should refer to the
             printed output to determine which parameter must be
             redefined.
   
        IFAILOverflow
             If the printed output before the overflow error contains a
             warning about serious ill-conditioning in the working set
             when adding the jth constraint, it may be possible to avoid
             the difficulty by increasing the magnitude of the optional
             parameter Linear Feasiblity Tolerance or Nonlinear
             Feasiblity Tolerance, and rerunning the program. If the
             message recurs even after this change, the offending
             linearly dependent constraint (with index 'j') must be
             removed from the problem. If overflow occurs in one of the
             user-supplied routines (e.g. if the nonlinear functions
             involve exponentials or singularities), it may help to
             specify tighter bounds for some of the variables (i.e.,
             reduce the gap between appropriate l  and u ).
                                                 j      j
   
        7. Accuracy
   
        If IFAIL = 0 on exit then the vector returned in the array X is
        an estimate of the solution to an accuracy of approximately
        Feasiblity Tolerance (see Section 5.1), whose default value is
                 0.8
        (epsilon)   , where (epsilon) is the machine precision (see
        X02AJF(*)).
   
        8. Further Comments
   
        In this section we give some further details of the method used
        by E04UCF.
   
        8.1. Solution of the Quadratic Programming Subproblem
   
        The search direction p is obtained by solving (4) using the
        method of E04NCF(*) (Gill et al [8]), which was specifically
        designed to be used within an SQP algorithm for nonlinear
        programming.
   
        The method of E04UCF is a two-phase (primal) quadratic
        programming method. The two phases of the method are: finding an
        initial feasible point by minimizing the sum of infeasibilities
        (the feasibility phase), and minimizing the quadratic objective
        function within the feasible region (the optimality phase). The
        computations in both phases are perfomed by the same subroutines.
        The two-phase nature of the algorithm is reflected by changing
        the function being minimized from the sum of infeasibilities to
        the quadratic objective function.
   
        In general, a quadratic program must be solved by iteration. Let
        p denote the current estimate of the solution of (4); the new
                _
        iterate p is defined by
   
                               _
                               p=p+(sigma)d,                          (9)
   
        where, as in (3), (sigma) is a non-negative step length and d is
        a search direction.
   
        At the beginning of each iteration of E04UCF, a working set is
        defined of constraints (general and bound) that are satisfied
        exactly. The vector d is then constructed so that the values of
        constraints in the working set remain unaltered for any move
        along d. For a bound constraint in the working set, this property
        is achieved by setting the corresponding component of d to zero,
        i.e., by fixing the variable at its bound. As before, the
        subscripts 'FX' and 'FR' denote selection of the components
        associated with the fixed and free variables.
   
        Let C denote the sub-matrix of rows of
   
                                      (A )
                                      ( L)
                                      (A )
                                      ( N)
   
        corresponding to general constraints in the working set. The
        general constraints in the working set will remain unaltered if
   
                                 C  d  =0,                           (10)
                                  FR FR
   
        which is equivalent to defining d   as
                                         FR
   
                                  d  =Zd                             (11)
                                   FR   z
   
        for some vector d , where Z is the matrix associated with the TQ
                         z
        factorization (5) of C  .
                              FR
   
        The definition of d  in (11) depends on whether the current p is
                           z
        feasible. If not, d  is zero except for a component (gamma) in
                           z
        the jth position, where j and (gamma) are chosen so that the sum
        of infeasibilities is decreasing along d. (For further details,
        see Gill et al [8].) In the feasible case, d  satisfies the
                                                    z
        equations
   
                               T       T
                              R R d =-Z q  ,                         (12)
                               z z z     FR
   
                                            T
        where R  is the Cholesky factor of Z H  Z and q is the gradient
               z                              FR
                                                                   T
        of the quadratic objective function (q=g+Hp). (The vector Z q
                                                                     FR
        is the projected gradient of the QP.) With (12), P+d is the
        minimizer of the quadratic objective function subject to treating
        the constraints in the working set as equalities.
   
        If the QP projected gradient is zero, the current point is a
        constrained stationary point in the subspace defined by the
        working set. During the feasiblity phase, the projected gradient
        will usually be zero only at a vertex (although it may vanish at
        non-vertices in the presence of constraint dependencies). During
        the optimality phase, a zero projected gradient implies that p
        minimizes the quadratic objective function when the constraints
        in the working set are treated as equalities. In either case,
        Lagrange multipliers are computed. Given a positive constant
        (delta) of the order of the machine precision, the Lagrange
        multiplier (mu)  corresponding to an inequality constraint in the
                       j
        working set at its upper bound is said to be optimal if
        (mu) <=(delta) when the jth constraint is at its upper bound, or
            j
        if (mu) >=-(delta) when the associated constraint is at its lower
               j
        bound. If any multiplier is non-optimal, the current objective
        function (either the true objective or the sum of
        infeasibilities) can be reduced by deleting the corresponding
        constraint from the working set.
   
        If optimal multipliers occur during the feasibility phase and the
        sum of infeasibilities is non-zero, no feasible point exists. The
        QP algorithm will then continue iterating to determine the
        minimum sum of infeasibilities. At this point, the Lagrange
        multiplier (mu)  will satisfy -(1+(delta))<=(mu) <=(delta) for an
                       j                                j
        inequality constraint at its upper bound, and
        -(delta)<=(mu) <=1+(delta) for an inequality at its lower bound.
                      j
        The Lagrange multiplier for an equality constraint will satisfy
        |(mu) |<=1+(delta).
             j
   
        The choice of step length (sigma) in the QP iteration (9) is
        based on remaining feasible with respect to the satisfied
        constraints. During the optimality phase, if p+d is feasible,
        (sigma) will be taken as unity. (In this case, the projected
                    _
        gradient at p will be zero.) Otherwise, (sigma) is set to
        (sigma) , the step to the 'nearest'constraint, which is added to
               M
        the working set at the next iteration.
   
        Each change in the working set leads to a simple change to C  :
                                                                    FR
        if the status of a general constraint changes, a row of C   is
                                                                 FR
        altered; if a bound constraint enters or leaves the working set,
        a column of C   changes. Explicit representations are recurred of
                     FR
                                                       T       T
        the matrices T, Q   and R, and of the vectors Q q and Q g.
                         FR
   
        8.2. The Merit Function
   
        After computing the search direction as described in Section 3,
        each major iteration proceeds by determining a step length
        (alpha) in (3) that produces a 'sufficient decrease' in the
        augmented Lagrangian merit function
   
   
                              --
        L(x,(lambda),s)=F(x)- > (lambda) (c (x)-s )
                              --        i  i     i
                              i
   
                              1 --                2
                            + - > (rho) (c (x)-s ) ,                 (13)
                              2 --     i  i     i
                                i
   
        where x, (lambda) and s vary during the linsearch. The summation
        terms in (13) involve only the nonlinear constraints. The vector
        (lambda) is an estimate of the Lagrange multipliers for the
        nonlinear constraints of (1). The non-negative slack variables
        {s } allow nonlinear inequality constraints to be treated without
          i
        introducing discontinuities. The solution of the QP subproblem
        (4) provides a vector triple that serves as a direction of search
        for the three sets of variables. The non-negative vector (rho) of
        penalty parameters is initialised to zero at the beginning of the
        first major iteration. Thereafter, selected components are
        increased whenever necessary to ensure descent for the merit
        function. Thus, the sequence of norms of (rho) (the printed
        quantity Penalty, see Section 5.2) is generally non-decreasing,
        although each (rho)  may be reduced a limited number of times.
                           i
   
        The merit function (13) and its global convergence properties are
        described in Gill et al [9].
   
        8.3. The Quasi-Newton Update
   
        The matrix H in (4) is a positive-definite quasi-Newton
        approximation to the Hessian of the Lagrangian function. (For a
        review of quasi-Newton methods, see Dennis and Schnabel [3].) At
                                                                     _
        the end of each major iteration, a new Hessian approximation H is
        defined as a rank-two modification of H. In E04UCF, the BFGS
        quasi-Newton update is used:
   
                          _     1     T    1   T
                          H=H- ----Hss H+ ---yy ,                    (14)
                                T          T
                               s Hs       y s
   
                _
        where s=x-x (the change in x).
   
        In E04UCF, H is required to be positive-definite. If H is
                           _
        positive-definite, H defined by (14) will be positive-definite if
                     T
        and only if y s is positive (see, e.g. Dennis and More [1]).
        Ideally, y in (14) would be taken as y , the change in gradient
                                              L
        of the Lagrangian function
   
                             _ _T         T
                          y =g-A (mu) -g+A (mu) ,                    (15)
                           L    N    N    N    N
   
        where (mu)  denotes the QP multipliers associated with the
                  N
                                                           T
        nonlinear constraints of the original problem. If y s is not
                                                           L
        sufficiently positive, an attempt is made to perform the update
        with a vector y of the form
   
                          m
                           N
                          --             _    _
                    y=y + >  (omega) (a (x)c (x)-a (x)c (x)),
                       L  --        i  i    i     i    i
                          i=1
   
        where (omega) >=0. If no such vector can be found, the update is
                     i
        perfomed with a scaled y ; in this case, M is printed to indicate
                                L
        that the update is modified.
   
        Rather than modifying H itself, the Cholesky factor of the
        transformed Hessian H  (6) is updated, where Q is the matrix from
                             Q
        (5) associated with the active set of the QP subproblem. The
        update (13) is equivalent to the following update to H :
                                                              Q
   
                     _        1        T     1     T
                     H =H - ------H s s H + ----y y ,                (16)
                      Q  Q   T     Q Q Q Q   T   Q Q
                            s H s           y s
                             Q Q Q           Q Q
   
                  T           T
        where y =Q y, and s =Q s. This update may be expressed as a rank-
               Q           Q
        one update to R (see Dennis and Schnabel [2]).
   
        9. Example
   
        This section describes one version of the so-called 'hexagon'
        problem (a different formulation is given as Problem 108 in Hock
        and Schittkowski [11]). The problem is to determine the hexagon
        of maximum area such that no two of its vertices are more than
        one unit apart (the solution is not a regular hexagon).
   
        All constraint types are included (bounds, linear, nonlinear),
        and the Hessian of the Lagrangian function is not positive-
        definite at the solution. The problem has nine variables, non-
        infinite bounds on seven of the variables, four general linear
        constraints, and fourteen nonlinear constraints.
   
        The objective function is
   
                      F(x)=-x x +x x -x x -x x +x x +x x .
                             2 6  1 7  3 7  5 8  4 9  3 8
   
        The bounds on the variables are
   
             x >=0, -1<=x <=1, x >=0,x >=0, x >=0,x <=0, and x <=0.
              1          3      5     6      7     8          9
   
        Thus,
   
                                                             T
                  l =(0,-infty,-1,-infty,0,0,0,-infty,-infty)
                   B
   
                                                               T
                 u =(infty,infty,1,infty,infty,infty,infty,0,0)
                  B
   
        The general linear constraints are
   
             x -x >=0,x -x >=0, x -x >=0,and x -x >=0.
              2  1     3  2      3  4         4  5
   
        Hence,
   
                  (0)     (-1  1 0  0  0 0 0 0 0)        (infty)
                  (0)     ( 0 -1 1  0  0 0 0 0 0)        (infty)
               l =(0), A =( 0  0 1 -1  0 0 0 0 0) and u =(infty).
                L (0)   L ( 0  0 0  1 -1 0 0 0 0)      L (infty)
   
        The nonlinear constraints are all of the form c (x)<=1, for
                                                       i
        i=1,2,...,14; hence, all components of l  are -infty, and all
                                                N
        components of u  are 1. The fourteen functions {c (x)} are
                       N                                 i
   
                    2  2
             c (x)=x +x ,
              1     1  6
   
                          2        2
             c (x)=(x -x ) +(x -x ) ,
              2      2  1     7  6
   
                          2  2
             c (x)=(x -x ) +x ,
              3      3  1    6
   
                          2        2
             c (x)=(x -x ) +(x -x ) ,
              4      1  4     6  8
   
                          2        2
             c (x)=(x -x ) +(x -x ) ,
              5      1  5     6  9
   
                    2  2
             c (x)=x +x ,
              6     2  7
   
                          2  2
             c (x)=(x -x ) +x ,
              7      3  2    7
   
                          2        2
             c (x)=(x -x ) +(x -x ) ,
              8      4  2     8  7
   
                          2        2
             c (x)=(x -x ) +(x -x ) ,
              9      2  5     7  9
   
                           2  2
             c  (x)=(x -x ) +x ,
              10      4  3    8
   
                           2  2
             c  (x)=(x -x ) +x ,
              11      5  3    9
   
                     2  2
             c  (x)=x +x ,
              12     4  8
   
                           2        2
             c  (x)=(x -x ) +(x -x ) ,
              13      4  5     9  8
   
                     2  2
             c  (x)=x +x .
              14     5  9
   
        An optimal solution (to five figures) is
   
   
         *
        x =(0.060947,0.59765,1.0,0.59765,0.060947,0.34377,0.5,
                         T
            -0.5,0.34377) ,
   
               *
        and F(x )=-1.34996. (The optimal objective function is unique,
        but is achieved for other values of x.) Five nonlinear
                                                        *
        constraints and one simple bound are active at x . The sample
        solution output is given later in this section, following the
        sample main program and problem definition.
   
        Two calls are made to E04UCF in order to demonstrate some of its
        features. For the first call, the starting point is:
   
                                                                      T
         x =(0.1,0.125,0.666666,0.142857,0.111111,0.2,0.25,-0.2,-0.25) .
          0
   
        All objective and constraint derivatives are specified in the
        user-provided subroutines OBJFN1 and CONFN1, i.e., the default
        option Derivative Level =3 is used.
   
        On completion of the first call to E04UCF, the optimal variables
        are perturbed to produce the initial point for a second run in
        which the problem functions are defined by the subroutines OBJFN2
        and CONFN2. To illustrate one of the finite-difference options in
        E04UCF, these routines are programmed so that the first six
        components of the objective gradient and the constant elements of
        the Jacobian matrix are not specified; hence, the option
        Derivative Level =0 is chosen. During computation of the finite-
        difference intervals, the constant Jacobian elements are
        identified and set, and E04UCF automatically increases the
        derivative level to 2.
   
        The second call to E04UCF illustrates the use of the Warm Start
        Level option to utilize the final active set, nonlinear
        multipliers and approximate Hessian from the first run. Note that
        Hessian = Yes was specified for the first run so that the array R
        would contain the Cholesky factor of the approximate Hessian of
        the Lagrangian.
   
        The two calls to E04UCF illustrate the alternative methods of
        assigning default parameters. (There is no special significance
        in the order of these assignments; an options file may just as
        easily be used to modify parameters set by E04UEF.)
   
        The results are typical of those obtained from E04UCF when
        solving well behaved (non-trivial) nonlinear problems. The
        approximate Hessian and working set remain relatively well-
        conditioned. Similarly the penalty parameters remain small and
        approximately constant. The numerical results illustrate much of
        the theoretically predicted behaviour of a quasi-Newton SQP
        method. As x approaches the solution, only one minor iteration is
        perfomed per major iteration, and the Norm Gz and Norm C columns
        exhibit the fast linear convergence rate mentioned in Sections 5
        and 6. Note that the constraint violations converge earlier than
        the projected gradient. The final values of the project gradient
        norm and constraint norm reflect the limiting accuracy of the two
        quantities. It is possible to achieve almost full precision in
        the constraint norm but only half precision in the projected
        gradient norm. Note that the final accuracy in the nonlinear
        constraints is considerably better than the feasibility
        tolerance, because the constraint violations are being refined
        during the last few iterations while the algorithm is working to
        reduce the projected gradient norm. In this problem, the
        constraint values and Lagrange multipliers at the solution are '
        well balanced', i.e., all the multipliers are approximately the
        same order of magnitude. The behaviour is typical of a well-
        scaled problem.
   
        The example program is not reproduced here. The source code for
        all example programs is distributed with the NAG Foundation
        Library software and should be available on-line.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

          E04 -- Minimizing or Maximizing a Function                 E04UDF
                  E04UDF -- NAG Foundation Library Routine Document

          Note: Before using this routine, please read the Users' Note for
          your implementation to check implementation-dependent details.
          The symbol (*) after a NAG routine name denotes a routine that is
          not included in the Foundation Library.

          1. Purpose

          To supply optional parameters to E04UCF from an external file.

          2. Specification

                 SUBROUTINE E04UDF (IOPTNS, INFORM)
                 INTEGER          IOPTNS, INFORM

          3. Description

          E04UDF may be used to supply values for optional parameters to
          E04UCF. E04UDF reads an external file and each line of the file
          defines a single optional parameter. It is only necessary to
          supply values for those parameters whose values are to be
          different from their default values.

          Each optional parameter is defined by a single character string
          of up to 72 characters, consisting of one or more items. The
          items associated with a given option must be separated by spaces,
          or equal signs (=). Alphabetic characters may be upper or lower
          case. The string

                 Print level = 1

          is an example of a string used to set an optional parameter. For
          each option the string contains one or more of the following
          items:

          (a)   A mandatory keyword.

          (b)   A phrase that qualifies the keyword.

          (c)   A number that specifies an INTEGER or real value. Such
                numbers may be up to 16 contiguous characters in Fortran
                77's I, F, E or D formats, terminated by a space if this is
                not the last item on the line.

          Blank strings and comments are ignored. A comment begins with an
          asterisk (*) and all subsequent characters in the string are
          regarded as part of the comment.

          The file containing the options must start with begin and must
          finish with end An example of a valid options file is:

                Begin * Example options file
                 Print level =10
                End

          Normally each line of the file is printed as it is read, on the
          current advisory message unit (see X04ABF), but printing may be
          suppressed using the keyword nolist To suppress printing of begin,
          nolist must be the first option supplied as in the file:

                Begin
                  Nolist
                  Print level = 10
                End

          Printing will automatically be turned on again after a call to
          E04UCF and may be turned on again at any time by the user by
          using the keyword list.

          Optional parameter settings are preserved following a call to
          E04UCF, and so the keyword defaults is provided to allow the user
          to reset all the optional parameters to their default values
          prior to a subsequent call to E04UCF.

          A complete list of optional parameters, their abbreviations,
          synonyms and default values is given in Section 5.1 of the
          document for E04UCF.

          4. References

          None.

          5. Parameters

           1:  IOPTNS -- INTEGER                                      Input
               On entry: IOPTNS must be the unit number of the options
               file. Constraint: 0 <= IOPTNS <= 99.

           2:  INFORM -- INTEGER                                     Output
               On exit: INFORM will be zero, if an options file with the
               current structure has been read. Otherwise INFORM will be
               positive. Positive values of INFORM indicate that an options
               file may not have been successfully read as follows:
               INFORM = 1
                     IOPTNS is not in the range [0,99].

               INFORM = 2
                     begin was found, but end-of-file was found before end
                     was found.

               INFORM = 3
                     end-of-file was found before begin was found.

          6. Error Indicators and Warnings

          If a line is not recognised as a valid option, then a warning
          message is output on the current advisory message unit (X04ABF).

          7. Accuracy

          Not applicable.

          8. Further Comments

          E04UEF may also be used to supply optional parameters to E04UCF.

          9. Example

          See the example for E04UCF.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

          E04 -- Minimizing or Maximizing a Function                 E04UEF
                  E04UEF -- NAG Foundation Library Routine Document

          Note: Before using this routine, please read the Users' Note for
          your implementation to check implementation-dependent details.
          The symbol (*) after a NAG routine name denotes a routine that is
          not included in the Foundation Library.

          1. Purpose

          To supply individual optional parameters to E04UCF.

          2. Specification

                 SUBROUTINE E04UEF (STRING)
                 CHARACTER*(*)    STRING

          3. Description

          E04UEF may be used to supply values for optional parameters to
          E04UCF. It is only necessary to call E04UEF for those parameters
          whose values are to be different from their default values. One
          call to E04UEF sets one parameter value.

          Each optional parameter is defined by a single character string
          of up to 72 characters, consisting of one or more items. The
          items associated with a given option must be separated by spaces,
          or equal signs (=). Alphabetic characters may be upper or lower
          case. The string

                Print level = 1

          is an example of a string used to set an optional parameter. For
          each option the string contains one or more of the following
          items:

          (a)   A mandatory keyword.

          (b)   A phrase that qualifies the keyword.

          (c)   A number that specifies an INTEGER or real value. Such
                numbers may be up to 16 contiguous characters in Fortran
                77's I, F, E or D formats, terminated by a space if this is
                not the last item on the line.

          Blank strings and comments are ignored. A comment begins with an
          asterisk (*) and all subsequent characters in the string are
          regarded as part of the comment.

          Normally, each user-specified option is printed as it is defined,
          on the current advisory message unit (see X04ABF), but this
          printing may be suppressed using the keyword nolist Thus the
          statement

                CALL E04UEF (`Nolist')

          suppresses printing of this and subsequent options. Printing will
          automatically be turned on again after a call to E04UCF, and may
          be turned on again at any time by the user, by using the keyword
          list.

          Optional parameter settings are preserved following a call to
          E04UCF, and so the keyword defaults is provided to allow the user
          to reset all the optional parameters to their default values by
          the statement,

                CALL E04UEF (`Defaults')

          prior to a subsequent call to E04UCF.

          A complete list of optional parameters, their abbreviations,
          synonyms and default values is given in Section 5.1 of the
          document for E04UCF.

          4. References

          None.

          5. Parameters

           1:  STRING -- CHARACTER*(*)                                Input
               On entry: STRING must be a single valid option string. See
               Section 3 above and Section 5.1 of the routine document for
               E04UCF. On entry: STRING must be a single valid option
               string. See Section 3 above and Section 5.1 of the routine
               document for E04UCF.

          6. Error Indicators and Warnings

          If the parameter STRING is not recognised as a valid option
          string, then a warning message is output on the current advisory
          message unit (X04ABF).

          7. Accuracy

          Not applicable.

          8. Further Comments

          E04UDF may also be used to supply optional parameters to E04UCF.

          9. Example

          See the example for E04UCF.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

          E04 -- Minimizing or Maximizing a Function                 E04YCF
                  E04YCF -- NAG Foundation Library Routine Document

          Note: Before using this routine, please read the Users' Note for
          your implementation to check implementation-dependent details.
          The symbol (*) after a NAG routine name denotes a routine that is
          not included in the Foundation Library.

          1. Purpose

          E04YCF returns estimates of elements of the variance-covariance
          matrix of the estimated regression coefficients for a nonlinear
          least squares problem. The estimates are derived from the
          Jacobian of the function f(x) at the solution.

          This routine may be used following any one of the nonlinear
          least-squares routines E04FCF(*), E04FDF, E04GBF(*), E04GCF,
          E04GDF(*), E04GEF(*), E04HEF(*), E04HFF(*).

          2. Specification

                 SUBROUTINE E04YCF (JOB, M, N, FSUMSQ, S, V, LV, CJ, WORK,
                1                   IFAIL)
                 INTEGER          JOB, M, N, LV, IFAIL
                 DOUBLE PRECISION FSUMSQ, S(N), V(LV,N), CJ(N), WORK(N)

          3. Description

          E04YCF is intended for use when the nonlinear least-squares
                          T
          function, F(x)=f (x)f(x), represents the goodness of fit of a
          nonlinear model to observed data. The routine assumes that the
          Hessian of F(x), at the solution, can be adequately approximated
               T
          by 2J J, where J is the Jacobian of f(x) at the solution. The
          estimated variance-covariance matrix C is then given by

                                 2  T  -1    T
                        C=(sigma) (J J)     J J non-singular,

                       2
          where (sigma)  is the estimated variance of the residual at the
                    

          solution, x, given by

                                               

                                          2  F(x)
                                   (sigma) = ----,
                                             m-n

          m being the number of observations and n the number of variables.

          The diagonal elements of C are estimates of the variances of the
          estimated regression coefficients. See the Chapter Introduction
          E04 and Bard [1] and Wolberg [2] for further information on the
          use of C.

                T
          When J J is singular then C is taken to be

                                           2  T  *
                                  C=(sigma) (J J) ,

                  T  *                           T
          where (J J)  is the pseudo-inverse of J J, but in this case the
          parameter IFAIL is returned as non-zero as a warning to the user
          that J has linear dependencies in its columns. The assumed rank
          of J can be obtained from IFAIL.

          The routine can be used to find either the diagonal elements of
          C, or the elements of the jth column of C, or the whole of C.

          E04YCF must be preceded by one of the nonlinear least-squares
          routines mentioned in Section 1, and requires the parameters
          FSUMSQ, S and V to be supplied by those routines. FSUMSQ is the
                                    

          residual sum of squares F(x), and S and V contain the singular
          values and right singular vectors respectively in the singular
          value decomposition of J. S and V are returned directly by the
          comprehensive routines E04FCF(*), E04GBF(*), E04GDF(*) and
          E04HEF(*), but are returned as part of the workspace parameter W
          from the easy-to-use routines E04FDF, E04GCF, E04GEF(*) and
          E04HFF(*). In the case of E04FDF, S starts at W(NS), where

                          NS=6*N+2*M+M*N+1+max(1,N*(N-1)/2)

          and in the cases of the remaining easy-to-use routines, S starts
          at W(NS), where

                     NS=7*N+2*M+M*N+N*(N+1)/2+1+max(1,N*(N-1)/2)

          The parameter V starts immediately following the elements of S,
          so that V starts at W(NV), where

                                      NV=NS+N.

          For all the easy-to-use routines the parameter LV must be
          supplied as N. Thus a call to E04YCF following E04FDF can be
          illustrated as


                 CALL E04FDF (M, N, X, FSUMSQ, IW, LIW, W, LW, IFAIL)
                 NS = 6*N + 2*M + M*N + 1 + MAX((1,(N*(N-1))/2)
                 NV = NS + N
                 CALL E04YCF (JOB, M, N, FSUMSQ, W(NS), W(NV),
                *             N, CJ, WORK, IFAIL)

                                                        2
          where the parameters M, N, FSUMSQ and the (n+n ) elements W(NS),
          WS(NS+1),..., W(NV+N*N-1) must not be altered between the calls
          to E04FDF and E04YCF. The above illustration also holds for a
          call to E04YCF following a call to one of E04GCF, E04GEF(*),
          E04HFF(*) except that NS must be computed as

                NS = 7*N + 2*M + M*N + (N*(N+1))/2 + 1 + MAX((1,N*(N-1))/2)

          4. References

          [1]   Bard Y (1974) Nonlinear Parameter Estimation. Academic
                Press.

          [2]   Wolberg J R (1967) Prediction Analysis. Van Nostrand.

          5. Parameters

           1:  JOB -- INTEGER                                         Input
               On entry: which elements of C are returned as follows:
               JOB = -1
                     The n by n symmetric matrix C is returned.

               JOB = 0
                     The diagonal elements of C are returned.

               JOB > 0
                     The elements of column JOB of C are returned.
                Constraint: -1 <= JOB <= N.

           2:  M -- INTEGER                                           Input
               On entry: the number m of observations (residuals f (x)).
                                                                  i
               Constraint: M >= N.

           3:  N -- INTEGER                                           Input
               On entry: the number n of variables (x ). Constraint: 1 <=
                                                     j
               N <= M.

           4:  FSUMSQ -- DOUBLE PRECISION                             Input
                                                                

               On entry: the sum of squares of the residuals, F(x), at the
                        

               solution x, as returned by the nonlinear least-squares
               routine. Constraint: FSUMSQ >= 0.0.

           5:  S(N) -- DOUBLE PRECISION array                         Input
               On entry: the n singular values of the Jacobian as returned
               by the nonlinear least-squares routine. See Section 3 for
               information on supplying S following one of the easy-to-use
               routines.

           6:  V(LV,N) -- DOUBLE PRECISION array               Input/Output
               On entry: the n by n right-hand orthogonal matrix (the
               right singular vectors) of J as returned by the nonlinear
               least-squares routine. See Section 3 for information on
               supplying V following one of the easy-to-use routines. On
               exit: when JOB >= 0 then V is unchanged.

               When JOB = -1 then the leading n by n part of V is
               overwritten by the n by n matrix C. When E04YCF is called
               with JOB = -1 following an easy-to-use routine this means
                                                             2
               that C is returned, column by column, in the n  elements of
                                                  2
               W given by W(NV),W(NV+1),...,W(NV+N -1). (See Section 3 for
               the definition of NV).

           7:  LV -- INTEGER                                          Input
               On entry:
               the first dimension of the array V as declared in the
               (sub)program from which E04YCF is called.
               When V is passed in the workspace parameter W following one
               of the easy-to-use least-square routines, LV must be the
               value N.

           8:  CJ(N) -- DOUBLE PRECISION array                       Output
               On exit: with JOB = 0, CJ returns the n diagonal elements
               of C.

               With JOB = j>0, CJ returns the n elements of the jth column
               of C.

               When JOB = -1, CJ is not referenced.

           9:  WORK(N) -- DOUBLE PRECISION array                  Workspace
               When JOB = -1 or 0 then WORK is used as internal workspace.

               When JOB > 0, WORK is not referenced.

          10:  IFAIL -- INTEGER                                Input/Output
               On entry: IFAIL must be set to 0, -1 or 1. Users who are
               unfamiliar with this parameter should refer to the Essential
               Introduction for details.

               On exit: IFAIL = 0 unless the routine detects an error or
               gives a warning (see Section 6).

               For this routine, because the values of output parameters
               may be useful even if IFAIL /=0 on exit, users are
               recommended to set IFAIL to -1 before entry. It is then
               essential to test the value of IFAIL on exit. To suppress
               the output of an error message when soft failure occurs, set
               IFAIL to 1.

          6. Error Indicators and Warnings

          Errors or warnings specified by the routine:

          IFAIL= 1
               On entry JOB < -1,

               or       JOB > N,

               or       N < 1,

               or       M < N,

               or       FSUMSQ < 0.0.

          IFAIL= 2
               The singular values are all zero, so that at the solution
               the Jacobian matrix J has rank 0.

          IFAIL> 2
               At the solution the Jacobian matrix contains linear, or near
               linear, dependencies amongst its columns. In this case the
               required elements of C have still been computed based upon J
               having an assumed rank given by (IFAIL-2). The rank is
               computed by regarding singular values SV(j) that are not
               larger than 10*(epsilon)*SV(1) as zero, where (epsilon) is
               the machine precision (see X02AJF(*)). Users who expect near
               linear dependencies at the solution and are happy with this
               tolerance in determining rank should call E04YCF with IFAIL
               = 1 in order to prevent termination by P01ABF(*). It is then
               essential to test the value of IFAIL on exit from E04YCF.

          IFAILOverflow
               If overflow occurs then either an element of C is very
               large, or the singular values or singular vectors have been
               incorrectly supplied.

          7. Accuracy

          The computed elements of C will be the exact covariances
          corresponding to a closely neighbouring Jacobian matrix J.

          8. Further Comments

          When JOB = -1 the time taken by the routine is approximately
                           3
          proportional to n . When JOB >= 0 the time taken by the routine
                                            2
          is approximately proportional to n .

          9. Example

          To estimate the variance-covariance matrix C for the least-
          squares estimates of x , x  and x  in the model
                                1   2      3

                                            t
                                             1
                                   y=x + ---------
                                      1  x t +x t
                                          2 2  3 3

          using the 15 sets of data given in the following table:

                                  y    t   t    t
                                        1   2    3
                                 0.14  1.0 15.0 1.0
                                 0.18  2.0 14.0 2.0
                                 0.22  3.0 13.0 3.0
                                 0.25  4.0 12.0 4.0
                                 0.29  5.0 11.0 5.0
                                 0.32  6.0 10.0 6.0
                                 0.35  7.0  9.0 7.0
                                 0.39  8.0  8.0 8.0
                                 0.37  9.0  7.0 7.0
                                 0.58 10.0  6.0 6.0
                                 0.73 11.0  5.0 5.0
                                 0.96 12.0  4.0 4.0
                                 1.34 13.0  3.0 3.0
                                 2.10 14.0  2.0 2.0
                                 4.39 15.0  1.0 1.0

          The program uses (0.5,1.0,1.5) as the initial guess at the
          position of the minimum and computes the least-squares solution
          using E04FDF. See the routine document E04FDF for further
          information.

          The example program is not reproduced here. The source code for
          all example programs is distributed with the NAG Foundation
          Library software and should be available on-line.

