Appendix D: Log-Likelihood Equations
This appendix covers the log-likelihood functions and their associated partial derivatives for most of the distributions available in Weibull++. These distributions are discussed in more detail in the chapter for each distribution.
Weibull Log-Likelihood Functions and their Partials
The Two-Parameter Weibull
This log-likelihood function is composed of three summation portions:
![{\displaystyle {\begin{aligned}\ln(L)=&\Lambda ={\underset {i=1}{\overset {{F}_{e}}{\mathop {\sum } }}}\,{{N}_{i}}\ln \left[{\frac {\beta }{\eta }}{{\left({\frac {{T}_{i}}{\eta }}\right)}^{\beta -1}}{{e}^{-{{\left({\tfrac {{T}_{i}}{\eta }}\right)}^{\beta }}}}\right]-{\underset {i=1}{\overset {S}{\mathop {\sum } }}}\,N_{i}^{\prime }{{\left({\frac {T_{i}^{\prime }}{\eta }}\right)}^{\beta }}\\&{\text{ }}+{\underset {i=1}{\overset {FI}{\mathop {\sum } }}}\,N_{i}^{\prime \prime }\ln \left[{{e}^{-{{\left({\tfrac {T_{Li}^{\prime \prime }}{\eta }}\right)}^{\beta }}}}-{{e}^{-{{\left({\tfrac {T_{Ri}^{\prime \prime }}{\eta }}\right)}^{\beta }}}}\right]\end{aligned}}\,\!}](https://en.wikipedia.org/api/rest_v1/media/math/render/svg/55b62e4e0a67bcbecbdfee940eeaa4d659e84e0f)
where:
-
is the number of groups of times-to-failure data points
is the number of times-to-failure in the
time-to-failure data group
is the Weibull shape parameter (unknown a priori, the first of two parameters to be found)
is the Weibull scale parameter (unknown a priori, the second of two parameters to be found)
is the time of the
group of time-to-failure data
is the number of groups of suspension data points
is the number of suspensions in
group of suspension data points
is the time of the
suspension data group
is the number of interval failure data groups
is the number of intervals in
group of data intervals
is the beginning of the
interval
is the ending of the
interval
For the purposes of MLE, left censored data will be considered to be intervals with
The solution will be found by solving for a pair of parameters
so that
and
It should be noted that other methods can also be used, such as direct maximization of the likelihood function, without having to compute the derivatives.
The Three-Parameter Weibull
This log-likelihood function is again composed of three summation portions:
![{\displaystyle {\begin{aligned}\ln(L)=&\Lambda ={\underset {i=1}{\overset {{F}_{e}}{\mathop {\sum } }}}\,{{N}_{i}}\ln \left[{\frac {\beta }{\eta }}{{\left({\frac {{{T}_{i}}-\gamma }{\eta }}\right)}^{\beta -1}}{{e}^{-{{\left({\tfrac {{{T}_{i}}-\gamma }{\eta }}\right)}^{\beta }}}}\right]-{\underset {i=1}{\overset {S}{\mathop {\sum } }}}\,N_{i}^{\prime }{{\left({\frac {T_{i}^{\prime }-\gamma }{\eta }}\right)}^{\beta }}\\&\\&+{\underset {i=1}{\overset {FI}{\mathop {\sum } }}}\,N_{i}^{\prime \prime }\ln \left[{{e}^{-{{\left({\tfrac {T_{Li}^{\prime \prime }-\gamma }{\eta }}\right)}^{\beta }}}}-{{e}^{-{{\left({\tfrac {T_{Ri}^{\prime \prime }-\gamma }{\eta }}\right)}^{\beta }}}}\right]\end{aligned}}\,\!}](https://en.wikipedia.org/api/rest_v1/media/math/render/svg/7fa57d66db7d7d8072d0b589ad087dcee3d1a5a6)
where:
-
is the number of groups of times-to-failure data points
is the number of times-to-failure in the
time-to-failure data group
is the Weibull shape parameter (unknown a priori, the first of three parameters to be found)
is the Weibull scale parameter (unknown a priori, the second of three parameters to be found)
is the time of the
group of time-to-failure data
is the Weibull location parameter (unknown a priori, the third of three parameters to be found)
is the number of groups of suspension data points
is the number of suspensions in
group of suspension data points
is the time of the
suspension data group
is the number of interval data groups
is the number of intervals in the
group of data intervals
is the beginning of the
interval- and
is the ending of the
interval
The solution is found by solving for
so that
and
It should be pointed out that the solution to the three-parameter Weibull via MLE is not always stable and can collapse if
In estimating the true MLE of the three-parameter Weibull distribution, two difficulties arise. The first is a problem of non-regularity and the second is the parameter divergence problem, as discussed in Hirose
[14].
Non-regularity occurs when 
In general, there are no MLE solutions in the region of
When
MLE solutions exist but are not asymptotically normal, as discussed in Hirose
[14]. In the case of non-regularity, the solution is treated anomalously.
Weibull++ attempts to find a solution in all of the regions using a variety of methods, but the user should be forewarned that not all possible data can be addressed. Thus, some solutions using MLE for the three-parameter Weibull will fail when the algorithm has reached predefined limits or fails to converge. In these cases, the user can change to the non-true MLE approach (in Weibull++ Application Setup), where
is estimated using non-linear regression. Once
is obtained, the MLE estimates of
and
are computed using the transformation
Exponential Log-Likelihood Functions and their Partials
The One-Parameter Exponential
This log-likelihood function is composed of three summation portions:
![{\displaystyle \ln(L)=\Lambda ={\underset {i=1}{\overset {{F}_{e}}{\mathop {\sum } }}}\,{{N}_{i}}\ln \left[\lambda {{e}^{-\lambda {{T}_{i}}}}\right]-{\underset {i=1}{\overset {S}{\mathop {\sum } }}}\,N_{i}^{\prime }\lambda T_{i}^{\prime }+{\underset {i=1}{\overset {FI}{\mathop {\sum } }}}\,N_{i}^{\prime \prime }\ln \left[{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}\right]\,\!}](https://en.wikipedia.org/api/rest_v1/media/math/render/svg/3a2fece73ad230e80be2572cb0c080abad8f8221)
where:
-
is the number of groups of times-to-failure data points
is the number of times-to-failure in the
time-to-failure data group
is the failure rate parameter (unknown a priori, the only parameter to be found)
is the time of the
group of time-to-failure data
is the number of groups of suspension data points
is the number of suspensions in the
group of suspension data points
is the time of the
suspension data group
is the number of interval data groups
is the number of intervals in the
group of data intervals
is the beginning of the
interval
is the ending of the
interval
The solution will be found by solving for a parameter
so that
Note that for
there exists a closed form solution.
![{\displaystyle {\begin{aligned}{\frac {\partial \Lambda }{\partial \lambda }}=&{\underset {i=1}{\overset {{F}_{e}}{\mathop {\sum } }}}\,{{N}_{i}}\left({\frac {1}{\lambda }}-{{T}_{i}}\right)-{\underset {i=1}{\overset {S}{\mathop {\sum } }}}\,N_{i}^{\prime }T_{i}^{\prime }\\&-{\underset {i=1}{\overset {FI}{\mathop {\sum } }}}\,N_{i}^{\prime \prime }\left[{\frac {T_{Li}^{\prime \prime }{{e}^{-\lambda T_{Li}^{\prime \prime }}}-T_{Ri}^{\prime \prime }{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}{{{e}^{-\lambda T_{Li}^{\prime \prime }}}-{{e}^{-\lambda T_{Ri}^{\prime \prime }}}}}\right]\end{aligned}}\,\!}](https://en.wikipedia.org/api/rest_v1/media/math/render/svg/0a0746f25fc77fa854576a8f28f9e12a296663d8)
The Two-Parameter Exponential
This log-likelihood function for the two-parameter exponential distribution is very similar to that of the one-parameter distribution and is composed of three summation portions:
![{\displaystyle {\begin{aligned}&\ln(L)=&\Lambda ={\underset {i=1}{\overset {{F}_{e}}{\mathop {\sum } }}}\,{{N}_{i}}\ln \left[\lambda {{e}^{-\lambda \left({{T}_{i}}-\gamma \right)}}\right]-{\underset {i=1}{\overset {S}{\mathop {\sum } }}}\,N_{i}^{\prime }\lambda \left(T_{i}^{\prime }-\gamma \right)\\&&\ \ +{\underset {i=1}{\overset {FI}{\mathop {\sum } }}}\,N_{i}^{\prime \prime }\ln \left[{{e}^{-\lambda \left(T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left(T_{Ri}^{\prime \prime }-\gamma \right)}}\right],\end{aligned}}\,\!}](https://en.wikipedia.org/api/rest_v1/media/math/render/svg/f9aa6946f26f37886cd09e2f33fea2881bcd94e7)
where:
-
is the number of groups of times-to-failure data points
is the number of times-to-failure in the
time-to-failure data group
is the failure rate parameter (unknown a priori, the first of two parameters to be found)
is the location parameter (unknown a priori, the second of two parameters to be found)
is the time of the
group of time-to-failure data
is the number of groups of suspension data points
is the number of suspensions in the
group of suspension data points
is the time of the
suspension data group
is the number of interval data groups
is the number of intervals in the
group of data intervals
is the beginning of the
interval
is the ending of the
interval
To find the two-parameter solution, look at the partial derivatives
and
:
![{\displaystyle {\begin{aligned}{\frac {\partial \Lambda }{\partial \lambda }}=&{\underset {i=1}{\overset {{F}_{e}}{\mathop {\sum } }}}\,{{N}_{i}}\left[{\frac {1}{\lambda }}-\left({{T}_{i}}-\gamma \right)\right]\\&-{\underset {i=1}{\overset {S}{\mathop {\sum } }}}\,N_{i}^{\prime }\left(T_{i}^{\prime }-\gamma \right)\\&-{\underset {i=1}{\overset {FI}{\mathop {\sum } }}}\,N_{i}^{\prime \prime }\left[{\frac {\left(T_{Li}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left(T_{Li}^{\prime \prime }-{{\gamma }_{0}}\right)}}-\left(T_{Ri}^{\prime \prime }-\gamma \right){{e}^{-\lambda \left(T_{Ri}^{\prime \prime }-\gamma \right)}}}{{{e}^{-\lambda \left(T_{Li}^{\prime \prime }-\gamma \right)}}-{{e}^{-\lambda \left(T_{Ri}^{\prime \prime }-\gamma \right)}}}}\right]\end{aligned}}\,\!}](https://en.wikipedia.org/api/rest_v1/media/math/render/svg/c8874eb7ba9b6d24077f0c37b7e0aaddcc64af22)
and
-
.
From here we see that 
is a positive, constant function of
. As alluded to in the chapter on the exponential distribution, this implies that the log-likelihood function
is, for fixed
, an increasing function of
. Thus the MLE for
is its largest possible value
. Therefore, to find the full MLE solution
for the two-parameter exponential distribution, one should set
equal to the first failure time and then find (numerically) a
such that
.
The 3D Plot utility in Weibull++ further illustrates this behavior of the log-likelihood function, as shown next:
Normal Log-Likelihood Functions and their Partials
The complete normal likelihood function (without the constant) is composed of three summation portions:
![{\displaystyle {\begin{aligned}\ln(L)=&\Lambda ={\underset {i=1}{\overset {{F}_{e}}{\mathop {\sum } }}}\,{{N}_{i}}\ln \left[{\frac {1}{\sigma }}\phi \left({\frac {{{T}_{i}}-\mu }{\sigma }}\right)\right]\\&+{\underset {i=1}{\overset {S}{\mathop {\sum } }}}\,N_{i}^{^{\prime }}\ln \left[1-\Phi \left({\frac {T_{i}^{^{\prime }}-\mu }{\sigma }}\right)\right]\\&{\text{ }}+{\underset {i=1}{\overset {{F}_{i}}{\mathop {\sum } }}}\,N_{i}^{^{\prime \prime }}\ln \left[\Phi \left({\frac {T_{{R}_{i}}^{^{\prime \prime }}-\mu }{\sigma }}\right)-\Phi \left({\frac {T_{{L}_{i}}^{^{\prime \prime }}-\mu }{\sigma }}\right)\right]\end{aligned}}\,\!}](https://en.wikipedia.org/api/rest_v1/media/math/render/svg/10b72499203307cdcf63c63097018b77e7b84f6b)
where:
-
is the number of groups of times-to-failure data points
is the number of times-to-failure in the
time-to-failure data group
is the mean parameter (unknown a priori, the first of two parameters to be found)
is the standard deviation parameter (unknown a priori, the second of two parameters to be found)
is the time of the
group of time-to-failure data
is the number of groups of suspension data points
is the number of suspensions in the
group of suspension data points
is the time of the
suspension data group
is the number of interval data groups
is the number of intervals in the
group of data intervals
is the beginning of the
interval
is the ending of the
interval
The solution will be found by solving for a pair of parameters
so that
and
where:
and:
Complete Data
Note that for the normal distribution, and in the case of complete data only (as was shown in Basic Statistical Background), there exists a closed-form solution for both of the parameters or:
and:
Lognormal Log-Likelihood Functions and their Partials
The general log-likelihood function (without the constant) for the lognormal distribution is composed of three summation portions:
![{\displaystyle {\begin{aligned}\ln(L)=&\Lambda ={\underset {i=1}{\overset {{F}_{e}}{\mathop {\sum } }}}\,{{N}_{i}}\ln \left[{\frac {1}{{{T}_{i}}{{\sigma }_{{T}'}}}}\phi \left({\frac {\ln \left({{T}_{i}}\right)-{\mu }'}{{\sigma }_{{T}'}}}\right)\right]\\&{\text{ }}+{\underset {i=1}{\overset {S}{\mathop {\sum } }}}\,N_{i}^{\prime }\ln \left[1-\Phi \left({\frac {\ln \left(T_{i}^{\prime }\right)-{\mu }'}{{\sigma }_{{T}'}}}\right)\right]\\&{\text{ }}+{\underset {i=1}{\overset {FI}{\mathop {\sum } }}}\,N_{i}^{\prime \prime }\ln \left[\Phi \left({\frac {\ln \left(T_{Ri}^{\prime \prime }\right)-{\mu }'}{{\sigma }_{{T}'}}}\right)-\Phi \left({\frac {\ln \left(T_{Li}^{\prime \prime }\right)-{\mu }'}{{\sigma }_{{T}'}}}\right)\right]\end{aligned}}\,\!}](https://en.wikipedia.org/api/rest_v1/media/math/render/svg/dbe2b6fa464c14bb4151298ef1373f3f4fefbe88)
where:
-
is the number of groups of times-to-failure data points
is the number of times-to-failure in the
time-to-failure data group
is the mean of the natural logarithms of the times-to-failure (unknown a priori, the first of two parameters to be found)
is the standard deviation of the natural logarithms of the times-to-failure (unknown a priori, the second of two parameters to be found)
is the time of the
group of time-to-failure data
is the number of groups of suspension data points
is the number of suspensions in the
group of suspension data points
is the time of the
suspension data group
is the number of interval data groups
is the number of intervals in the
group of data intervals
is the beginning of the
interval
is the ending of the
interval
The solution will be found by solving for a pair of parameters
so that
and
:
where:
and:
Mixed Weibull Log-Likelihood Functions and their Partials
The log-likelihood function (without the constant) is composed of three summation portions:
![{\displaystyle {\begin{aligned}\ln(L)=&\Lambda ={\underset {i=1}{\overset {{F}_{e}}{\mathop {\sum } }}}\,{{N}_{i}}\ln \left[{\underset {k=1}{\overset {Q}{\mathop {\sum } }}}\,{{\rho }_{k}}{\frac {{\beta }_{k}}{{\eta }_{k}}}{{\left({\frac {{T}_{i}}{{\eta }_{k}}}\right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left({\tfrac {{T}_{i}}{{\eta }_{k}}}\right)}^{{\beta }_{k}}}}}\right]\\&{\text{ }}+{\underset {i=1}{\overset {S}{\mathop {\sum } }}}\,N_{i}^{\prime }\ln \left[{\underset {k=1}{\overset {Q}{\mathop {\sum } }}}\,{{\rho }_{k}}{{e}^{-{{\left({\tfrac {T_{i}^{\prime }}{{\eta }_{k}}}\right)}^{{\beta }_{k}}}}}\right]\\&{\text{ }}+{\underset {i=1}{\overset {FI}{\mathop {\sum } }}}\,N_{i}^{\prime \prime }\ln \left[{\underset {k=1}{\overset {Q}{\mathop {\sum } }}}\,{{\rho }_{k}}{\frac {{\beta }_{k}}{{\eta }_{k}}}{{\left({\frac {T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}}}\right)}^{{{\beta }_{k}}-1}}{{e}^{-{{\left({\tfrac {T_{Li}^{\prime \prime }+T_{Ri}^{\prime \prime }}{2{{\eta }_{k}}}}\right)}^{{\beta }_{k}}}}}\right]\end{aligned}}\,\!}](https://en.wikipedia.org/api/rest_v1/media/math/render/svg/29ada02d127ad5767d88ecaaa8c2aa9d5d236db8)
where:
-
is the number of groups of times-to-failure data points
is the number of times-to-failure in the
time-to-failure data group
is the number of subpopulations
is the proportionality of the
subpopulation (unknown a priori, the first set of three sets of parameters to be found)
is the Weibull shape parameter of the
subpopulation (unknown a priori, the second set of three sets of parameters to be found)
is the Weibull scale parameter (unknown a priori, the third set of three sets of parameters to be found)
is the time of the
group of time-to-failure data
is the number of groups of suspension data points
is the number of suspensions in
group of suspension data points
is the time of the
suspension data group
is the number of groups of interval data points
is the number of intervals in
group of data intervals
is the beginning of the
interval
is the ending of the
interval
The solution will be found by solving for a group of parameters:
so that:
Logistic Log-Likelihood Functions and their Partials
This log-likelihood function is composed of three summation portions:
where:
-
is the number of groups of times-to-failure data points
is the number of times-to-failure in the
time-to-failure data group
is the logistic shape parameter (unknown a priori, the first of two parameters to be found)
is the logistic scale parameter (unknown a priori, the second of two parameters to be found)
is the time of the
group of time-to-failure data
is the number of groups of suspension data points
is the number of suspensions in
group of suspension data points
is the time of the
suspension data group
is the number of interval failure data group
is the number of intervals in
group of data intervals
is the beginning of the
interval
is the ending of the
interval
For the purposes of MLE, left censored data will be considered to be intervals with
The solution of the maximum log-likelihood function is found by solving for (
so that
The Loglogistic Log-Likelihood Functions and their Partials
This log-likelihood function is composed of three summation portions:
where:
-
is the number of groups of times-to-failure data points
is the number of times-to-failure in the
time-to-failure data group
is the loglogistic shape parameter (unknown a priori, the first of two parameters to be found)
is the loglogistic scale parameter (unknown a priori, the second of two parameters to be found)
is the time of the
group of time-to-failure data
is the number of groups of suspension data points
is the number of suspensions in
group of suspension data points
is the time of the
suspension data group
is the number of interval failure data groups,
is the number of intervals in
group of data intervals
is the beginning of the
interval
is the ending of the
interval
For the purposes of MLE, left censored data will be considered to be intervals with
The solution of the maximum log-likelihood function is found by solving for (
so that
The Gumbel Log-Likelihood Functions and their Partials
This log-likelihood function is composed of three summation portions:
or:
where:
-
is the number of groups of times-to-failure data points
is the number of times-to-failure in the
time-to-failure data group
is the Gumbel shape parameter (unknown a priori, the first of two parameters to be found)
is the Gumbel scale parameter (unknown a priori, the second of two parameters to be found)
is the time of the
group of time-to-failure data
is the number of groups of suspension data points
is the number of suspensions in
group of suspension data points
is the time of the
suspension data group
is the number of interval failure data groups
is the number of intervals in
group of data intervals
is the beginning of the
interval
is the ending of the
interval
For the purposes of MLE, left censored data will be considered to be intervals with
The solution of the maximum log-likelihood function is found by solving for (
so that:
The Gamma Log-Likelihood Functions and their Partials
This log-likelihood function is composed of three summation portions:
or:
where:
-
is the number of groups of times-to-failure data points
is the number of times-to-failure in the
time-to-failure data group
is the gamma shape parameter (unknown a priori, the first of two parameters to be found)
is the gamma scale parameter (unknown a priori, the second of two parameters to be found)
is the time of the
group of time-to-failure data
is the number of groups of suspension data points
is the number of suspensions in
group of suspension data points
is the time of the
suspension data group
is the number of interval failure data groups
is the number of intervals in
group of data intervals
is the beginning of the
interval- and
is the ending of the
interval
For the purposes of MLE, left censored data will be considered to be intervals with
The solution of the maximum log-likelihood function is found by solving for (
so that
