Chapter 10
Optimization functionslibrary
The
optimization function library
contains routines for maximum likelihood estimation of econometric models. Given a likelihood function whichdepends on the data as well as the parameters of the estimation problem,these routines will ﬁnd a set of parameter values that maximize the likelihoodfunction. In addition to discussing and illustrating use of the functions in the
optimization function library
, we also show how to use the MATLABsimplex optimization algorithms
fmin
and
fmins
as well as Em algorithmsto solve for maximum likelihood estimates in econometric models. An example that demonstrates how to incorporate a new optimization algorithmobtained from the Internet into the library is also provided.Unlike Gauss software, MATLAB does not contain a single ‘maxlik’ function that serves as an ‘all purpose’ routine for solving econometric maximumlikelihood optimization problems. In an attempt to remedy this, a function
maxlik
was devised for MATLAB. Unfortunately, this function is nowherenear as robust or professionally crafted as the Gauss function by the samename. This function as well as univariate and multivariate simplex routines,algorithms from
Numerical Recipes
that have been recoded for MATLAB,some Internet optimization functions written for MATLAB and a generalpurpose function
minz
contributed by Mike Cliﬀ form the basis of the optimization library,.The MathWorks sells an optimization toolbox that contains a host of algorithms for solving constrained and unconstrained optimization problems.These functions are not oriented toward econometric models, but rathersolution of very general optimization problems. To use these routines in260
CHAPTER 10. OPTIMIZATION FUNCTIONS LIBRARY
261solving econometric optimization problems, an interface similar to that described in Section 10.3 would need to be developed to adapt the algorithmsfor use in an econometric likelihood function setting.Section 10.1 illustrates the use of a univariate simplex algorithm
fmin
from the MATLAB toolbox. This type of algorithm is useful for cases wherethe optimization problem involves a single parameter, a case often encountered with concentrated likelihood functions. It also demonstrates the useof a multivariate simplex optimization function
fmins
from the MATLABtoolbox.Section 10.2 demonstrates use of the EMalgorithm approach to solvingmaximum likelihood estimation problems. Finally, Section 10.3 deals withthe most general case where the likelihood function is optimized with respectto a multivariate vector of parameters using numerical gradient methods.
10.1 Simplex optimization
Simplex optimization has an advantage in that it does not require computation of the derivatives of the function being optimized. It also has adisadvantage in that the usual estimates of dispersion for the parametersin the model obtained from the numerical hessian matrix in gradient methods are not available. The MATLAB toolbox contains both a univariatesimplex algorithm
fmin
and a multivariate function
fmins
. The use of theunivariate function is taken up in Section 10.1.1 and
fmins
is illustrated inSection 10.1.2.
10.1.1 Univariate simplex optimization
As an example of using the MATLAB
fmin
univariate simplex optimization function to solve an econometric maximum likelihood estimation problem, consider the BoxCox model. This model relies on a transformationof the sample data vector
y
(
λ
)
= (
y
λ
−
1)
/λ
for the case where
λ
= 0and
y
(
λ
)
=
ln
(
y
) when
λ
= 0. The model may also involve an identicaltransformation of the explanatory variables matrix, which we denote
X
(
λ
)
.A more general model involves diﬀerent values,
λ
1
,λ
2
associated with the
y
(
λ
1
)
transformation and
X
(
λ
2
)
transformations, and a very general modelallows for individual
λ
xi
,i
= 1
,...,k
for each explanatory variable vector inthe sample data matrix
X
.We focus initially on the simple model involving a single parameter
λ
.This model produces a loglinear model when
λ
1
=
λ
2
= 0, a semilogmodel when
λ
1
= 1
,λ
2
= 0, as well as more general ﬂexible functional forms
CHAPTER 10. OPTIMIZATION FUNCTIONS LIBRARY
262associated with values like,
λ
= 1
/
2, or
λ
=
√
2. Some of the functionalforms associated with values for
λ
1
,λ
2
are:
ln y
i
=
β
0
+
ln X
i
β
+
ε
i
, λ
1
= 0
,λ
2
= 0 (10.1)
y
i
=
β
0
+
ln X
i
β
+
ε
i
, λ
1
= 1
,λ
2
= 0 (10.2)
y
i
=
β
0
+
X
i
β
+
ε
i
, λ
1
= 1
,λ
2
= 1 (10.3)The log likelihood function is concentrated with respect to
β
(
λ
)
,σ
(
λ
),producing a function of only a single parameter
λ
:
L
(
λ

y,X
) = const + (
λ
−
1)
n
i
=1
ln y
i
−
(
n/
2)
ln
ˆ
σ
2
(
λ
) (10.4)where:ˆ
σ
2
(
λ
) = [
y
(
λ
)
−
X
(
λ
)
ˆ
β
(
λ
)
(
y
(
λ
)
−
X
(
λ
)
ˆ
β
(
λ
)]
/n
(10.5)ˆ
β
(
λ
) = (
X
(
λ
)
X
(
λ
)
)
−
1
X
(
λ
)
y
(
λ
)
To minimize the log of this likelihood function with respect to the parameter
λ
we can use the MATLAB function
fmin
that implements a simplexoptimization procedure. This function also allows us to set lower and upperlimits on the parameter
λ
to which the simplex search will be constrained.It is often the case that, values of
−
2
≤
λ
≤
2 are thought to representa reasonable range of feasible values for the parameter
λ
in the BoxCoxmodel.Our ﬁrst task in using
fmin
is to write a loglikelihood function thatevaluates the concentrated loglikelihood for any value of the parameter
λ
and returns a scalar value equal to the negative of the loglikelihood function.(Minimizing the negative loglikelihood is equivalent to maximizing the loglikelihood.) This function is shown below:
function like = box_lik(lam,y,x,model);% PURPOSE: evaluate BoxCox model concentrated likelihood function%% USAGE: like = box_lik(lam,y,x,model)% where: lam = boxcox parameter (scalar)% y = dependent variable vector (untransformed)% x = explanatory variables matrix (untransformed)% model = 0 for ytransform only, 1 for y,x both transformed
CHAPTER 10. OPTIMIZATION FUNCTIONS LIBRARY
263
% NOTE: x should contain intercept vector in 1st column (if desired)%% RETURNS: like = (a scalar) = log likelihood function%[n k] = size(x); ys = boxc_trans(y,lam);if model == 1 % user wants to transform both y,x% see if an intercept term exists in the modeliota = x(:,1); ifind = find(iota == 1);if isempty(ifind), xs = boxc_trans(x,lam); % no interceptelse, if length(ifind) == n, % an interceptxtrans = boxc_trans(x(:,2:k),lam);xs = [ones(n,1) xtrans];else, xs = boxc_trans(x,lam); % no interceptend;end;elseif model == 0, xs = x; % transform only yvectorend;bhat = inv(xs’*xs)*xs’*ys; e = ys  xs*bhat; sige = (e’*e)/n;like = (lam  1)*sum(log(y)) (n/2)*log(sige); like = like;
The function relies on another function
boxc trans
to carry out theBoxCox data transformation. It also contains an argument ‘model’ thatallows for a case (‘model=0’) where the
y
variable alone is transformed andanother case (‘model=1’) where both
y
and the
X
variables are transformed.The function
boxc trans
is:
function z = boxc_trans(x,lam)% PURPOSE: compute boxcox transformation%% USAGE: bdata = boxc_trans(data,lam)% where: lam = scalar transformation parameter% data = matrix nobs x k%% RETURNS: bdata = data matrix boxcox transformed[n k] = size(x); z = zeros(n,k); iota = ones(n,1);for i=1:k;if lam ~= 0, z(:,i) = (x(:,i).^lam  iota)/lam;else, z(:,i) = log(x(:,i)); end;end;
Now we can turn attention to use of the MATLAB simplex optimizationfunction
fmin
, which can be called with: a string containing the functionname ‘box lik’, an upper and lower limit for the simplex optimization searchover values of
λ
(‘lamlo’,‘lamup’), a 4x1 vector of optimization options (‘foptions’), and arguments that will be passed along to our likelihood function‘box lik’. In this example, we wish to pass ‘y,x,model’ as arguments to the‘box lik’ function. An example of the call is: