| NEWS | R Documentation |
change gradient testing from absolute to relative
This version incorporates no changes in functionality, just modifications to testing and dependencies for CRAN/backward compatibility.
change drop1 example to prevent use of old/incompatible
pbkrtest versions, for 2.15.3 compatibility
explicitly require(mlmRev) for tests to prevent cyclic
dependency
bump RcppEigen Imports: requirement from >0.3.1.2.3 to
>=0.3.2.0; Rcpp dependency to >= 0.10.5
improved NA handling in simulate and refit
made internal handling of weights/offset
arguments slightly more robust (Github #191)
handle non-positive-definite estimated fixed effect
variance-covariance matrices slightly more generally/robustly
(fall back on RX approximation, with a warning,
if finite-difference Hessian
is non-PD; return NA matrix if RX approximation is
also bad)
Added output specifying when Gauss-Hermite quadrature was used to fit the model, and specifying number of GHQ points (Github #190)
Models with prior weights returned an incorrect sigma and deviance (Github issue #155). The deviance bug was only a practical issue in model comparisons, not with inferences given a particular model. Both bugs are now fixed.
Profiling failed in some cases for models with vector random effects (Github issue #172)
Standard errors of fixed effects are now computed
from the approximate Hessian by default (see the
use.hessian argument in vcov.merMod); this
gives better (correct) answers when the estimates of
the random- and fixed-effect parameters are correlated
(Github #47)
The default optimizer for lmer fits has been
switched from "Nelder_Mead" to "bobyqa" because we have
generally found the latter to be more reliable. To switch
back to the old behaviour,
use control=lmerControl(optimizer="Nelder_Mead").
Better handling of rank-deficient/overparameterized
fixed-effect model matrices; see check.rankX option
to [g]lmerControl. The default value is
"message+drop.cols", which automatically drops redundant
columns and issues a message (not a warning). (Github #144)
slight changes in convergence checking; tolerances can
be specified where appropriate, and some default tolerances
have changed (e.g., check.conv.grad)
improved warning messages about rank-deficiency in X and Z etc. (warnings now try to indicate whether the unidentifiability is in the fixed- or random-effects part of the model)
predict and simulate now prefer
re.form as the argument to specify which random effects
to condition on, but allow ReForm, REForm, or
REform, giving a message (not a warning) that they are
deprecated (addresses Github #170)
small fixes for printing consistency in models with no fixed effects
we previously exported a fortify function identical
to the one found in ggplot2 in order to be able to define a
fortify.merMod S3 method without inducing a dependency on
ggplot2. This has now been unexported to avoid masking
ggplot2's own fortify methods; if you want to
add diagnostic information to the results of a model, use
fortify.merMod explicitly.
simulate.formula now checks for names associated
with the theta and beta parameter vectors. If
missing, it prints a message (not a warning); otherwise, it
re-orders the parameter vectors to match the internal
representation.
preliminary implementation of a check.scaleX argument
in [g]lmerControl that warns about scaling if some columns
of the fixed-effect model matrix have large standard
deviations (relative to 1, or to each other)
The gradient and Hessian are now computed via finite
differencing after the nonlinear fit is done, and the results
are used for additional convergence tests. Control of the
behaviour is available through the check.conv.* options
in [g]lmerControl. Singular fits (fits with estimated
variances of zero or correlations of +/- 1) can also be tested for,
although the current default value of the check.conv.singular
option is "ignore"; this may be changed to "warning"
in the future. The results are stored in @optinfo$derivs.
(Github issue #120; based on code by Rune Christensen.)
The simulate method will now work to generate
simulations "from scratch" by providing a model formula,
a data frame holding the predictor variables, and a list
containing the values of the model parameters:
see ?simulate.merMod. (Github issue #115)
VarCorr.merMod objects now have an as.data.frame
method, converting the list of matrices to a more
convenient form for reporting and post-processing. (Github issue #129)
results of fitted(), predict(),
and residuals() now have
names in all cases (previously results were unnamed, or
named only when predicting from new data)
the anova method now has a refit argument
that controls whether objects of class lmerMod should be
refitted with ML before producing the anova table.
(Github issues #141, #165; contributed by Henrik Singmann.)
the print method for VarCorr objects
now has a formatter argument for finer control
of standard deviation and variance formats
the optinfo slot now stores slightly more
information, including the number of function evaluations
($feval).
dotplot.ranef.mer now adds titles to sub-plots by default,
like qqmath.ranef.mer
fitted now respects na.action settings (Github
issue #149)
confint(.,method="boot") now works when there are
NA values in the original data set (Github issue #158)
previously, the code stored the results (parameter values, residuals, etc.) based on the last set of parameters evaluated, rather than the optimal parameters. These were not always the same, but were almost always very close, but some previous results will change slightly (Github issue #166)
when using the default method="profile",
confint now returns appropriate upper/lower bounds
(-1/1 for correlations, 0/Inf for standard deviations)
rather than NA when appropriate
in a previous development version, ranef returned
incorrect conditional variances (github issue #148). this is
now fixed
prediction now works when new data have fewer factor levels than are present in the original data (Github issue #143, reported by Rune Haubo)
the existence of a variable "new" in the global environment
would mess lme4 up: reported at http://stackoverflow.com/questions/19801070/error-message-glmer-using-r-what-must-be-a-character-string-or-a-function
confint.merMod and vcov.merMod are
now exported, for downstream package-author convenience
the package now depends on Matrix >=1.1-0 and RcppEigen >=0.3.1.2.3
new rename.response option for refit (see BUG
FIXES section)
eliminated redundant messages about suppressed fixed-effect correlation matrices when p>20
most inverse-link functions are now bounded where
appropriate by .Machine$double.eps, allowing fitting
of GLMMs with extreme parameter values
merMod objects created with refit did not
work with update: optional
rename.response option added to refit.merMod, to allow
this (but the default is still FALSE, for
back-compatibility) (reported by A. Kuznetsova)
fixed buglet preventing on-the-fly creation of index variables,
e.g. y~1+(1|rownames(data)) (reported by J. Dushoff)
predict now works properly for glmer models
with basis-creating terms (e.g. poly, ns)
step sizes determined from fixed effect coefficient standard
errors after first state of glmer fitting are now bounded,
allowing some additional models to be fitted
refit() now works, again, with lists of
length 1, so that e.g. refit(.,simulate(.)) works.
(Reported by Gustaf Granath)
getME(.,"ST") was returning a list
containing the Cholesky factorizations that get repeated in
Lambda. But this was inconsistent with what ST represents in
lme4.0. This inconsistency has now been fixed and
getME(.,"ST") is now consistent with the definition of the
ST matrix in lme4.0. See
https://github.com/lme4/lme4/issues/111 for more
detail. Thanks to Vince Dorie.
Corrected order of unpacking of standard
deviation/correlation components, which affected results
from confint(.,method="boot"). (Reported by Reinhold
Kliegl)
fixed a copying bug that made refitML()
modify the original model
check.numobs.* and check.numlev.* in
(g)lmerControl have been changed (from recent development
versions) to check.nobs.* and
check.nlev.* respectively, and the default values of
check.nlev.gtreq.5 and check.nobs.vs.rankZ
have been changed to "ignore" and "warningSmall"
respectively
in (g)lmerControl, arguments to the optimizer
should be passed as a list called optCtrl, rather than
specified as additional (ungrouped) arguments
the postVar argument to ranef has been
changed to the (more sensible) condVar ("posterior variance"
was a misnomer, "conditional variance" – short for "variance of the
conditional mode" – is preferred)
the REform argument to predict has been changed
to ReForm for consistency
the tnames function, briefly exported, has been
unexported
getME(.,"cnms") added
print method for merMod objects is now more
terse, and different from summary.merMod
the objective method for the respMod
reference class now takes an optional sigma.sq parameter
(defaulting to NULL) to allow calculation of the
objective function with a residual variance different from
the profiled value (Vince Dorie)
Because the internal computational machinery has changed,
results from the newest version of lme4 will not be numerically
identical to those from previous versions. For reasonably well-
defined fits, they will be extremely close (within numerical
tolerances of 1e-4 or so), but for unstable or poorly-defined fits
the results may change, and very unstable fits may fail when they
(apparently) succeeded with previous versions. Similarly, some fits
may be slower with the new version, although on average the new
version should be faster and more stable. More numerical
tuning options are now available (see below); non-default settings
may restore the speed and/or ability to fit a particular model without
an error. If you notice significant or disturbing changes when fitting
a model with the new version of lme4, please notify the maintainers.
VarCorr returns its results in the same format as before (as a
list of variance-covariance matrices with correlation and stddev
attributes, plus a sc attribute giving the residual standard
deviation/scale parameter when appropriate), but prints them in a
different (nicer) way.
By default residuals gives deviance (rather than Pearson)
residuals when applied to glmer fits (a side effect of matching glm
behaviour more closely).
As another side effect of matching glm
behaviour, reported log-likelihoods from glmer models
are no longer consistent with those from pre-1.0 lme4,
but are consistent with glm; see glmer
examples.
More use is made of S3 rather than S4 classes and methods: one
side effect is that the nlme and lme4 packages are now much more
compatible; methods such as fixef no longer conflict.
The internal optimizer has changed. [gn]lmer now has an
optimizer argument; "Nelder_Mead" is the default for [n]lmer,
while a combination of "bobyqa" (an alternative derivative-free
method) and "Nelder_Mead" is the default for glmer. To use the
nlminb optimizer as in the old version of lme4, you can use
optimizer="optimx" with control=list(method="nlminb") (you will
need the optimx package to be installed and loaded). See
lmerControl for details.
Families in GLMMs are no longer restricted to built-in/hard-
coded families; any family described in family, or following that
design, is usable (although there are some hard-coded families, which
will be faster).
[gn]lmer now produces objects of class merMod rather than
class mer as before.
the structure of the Zt (transposed random effect
design matrix) as returned by getME(.,"Zt"), and the
corresponding order of the random effects vector
(getME(.,"u")) have changed. To retrieve Zt
in the old format, use do.call(Matrix::rBind,getME(.,"Ztlist")).
the package checks input more thoroughly for
non-identifiable or otherwise problematic cases: see
lmerControl for fine control of the test behaviour.
A general-purpose getME accessor method allows
extraction of a wide variety of components of a mixed-model
fit. getME also allows a vector of objects to be returned as
a list of mixed-model components. This has been backported to
be compatible with older versions of lme4 that still produce mer
objects rather than merMod objects. However, backporting is incomplete;
some objects are only extractable in newer versions of lme4.
Optimization information (convergence codes, warnings, etc.)
is now stored in an @optinfo slot.
bootMer provides a framework for obtaining parameter confidence
intervals by parametric bootstrapping.
plot.merMod provides diagnostic plotting
methods similar to those from the nlme package
(although missing augPred).
A predict.merMod method gives predictions;
it allows an effect-specific choice of conditional prediction or prediction at the
population level (i.e., with random effects set to zero).
Likelihood profiling for lmer and glmer results (see
link{profile-methods}).
Confidence intervals by likelihood profiling (default),
parametric bootstrap, or Wald approximation (fixed effects only):
see confint.merMod
nAGQ=0, an option to do fast (but inaccurate) fitting of GLMMs.
Using devFunOnly=TRUE allows the user to extract a deviance
function for the model, allowing further diagnostics/customization of
model results.
The internal structure of [gn]lmer is now more modular, allowing
finer control of the different steps of argument checking; construction
of design matrices and data structures; parameter estimation; and construction
of the final merMod object (see ?modular).
the formula, model.frame, and terms
methods return full versions (including random effect terms and
input variables) by default, but a fixed.only argument
allows access to the fixed effect submodel.
glmer.nb provides an embryonic negative
binomial fitting capability.
Adaptive Gaussian quadrature (AGQ) is not available for multiple and/or non-scalar random effects.
Posterior variances of conditional models for non-scalar random effects.
Standard errors for predict.merMod results.
Automatic MCMC sampling based on the fit turns out to be very difficult
to implement in a way that is really broadly reliable and robust; mcmcsamp
will not be implemented in the near future. See
pvalues for alternatives.
"R-side" structures (within-block correlation and heteroscedasticity) are not on the current timetable.
In a development version, prior weights were not being used properly in the calculation of the residual standard deviation, but this has been fixed. Thanks to Simon Wood for pointing this out.
In a development version, the step-halving component of the penalized iteratively reweighted least squares algorithm was not working, but this is now fixed.
In a development version, square RZX matrices would lead to a
pwrssUpdate did not converge in 30 iterations error. This has been fixed
by adding an extra column of zeros to RZX.
Previous versions of lme4 provided
the mcmcsamp function, which efficiently generated
a Markov chain Monte Carlo sample from the posterior
distribution of the parameters, assuming flat (scaled
likelihood) priors. Due to difficulty in constructing a
version of mcmcsamp that was reliable even in
cases where the estimated random effect variances were
near zero (e.g.
https://stat.ethz.ch/pipermail/r-sig-mixed-models/2009q4/003115.html),
mcmcsamp has been withdrawn (or more precisely,
not updated to work with lme4 versions >=1.0).
Calling glmer with the default gaussian family
redirects to lmer, but this is deprecated
(in the future glmer(...,family="gaussian") may
fit a LMM using the penalized iteratively reweighted least squares
algorithm). Please call lmer directly.
Calling lmer with a family argument redirects
to glmer; this is deprecated. Please call glmer directly.
The underlying algorithms and representations for all the mixed-effects models fit by this package have changed - for the better, we hope. The class "mer" is a common mixed-effects model representation for linear, generalized linear, nonlinear and generalized nonlinear mixed-effects models.
ECME iterations are no longer used at all, nor are analytic gradients. Components named 'niterEM', 'EMverbose', or 'gradient' can be included in the 'control' argument to lmer(), glmer() or nlmer() but have no effect.
PQL iterations are no longer used in glmer() and nlmer(). Only the Laplace approximation is currently available. AGQ, for certain classes of GLMMs or NLMMs, is being added.
The 'method' argument to lmer(), glmer() or nlmer() is deprecated. Use the 'REML = FALSE' in lmer() to obtain ML estimates. Selection of AGQ in glmer() and nlmer() will be controlled by the argument 'nAGQ', when completed.
The representation of mixed-effects models has been dramatically changed to allow for smooth evaluation of the objective as the variance-covariance matrices for the random effects approach singularity. Beta testers found this representation to be more robust and usually faster than previous versions of lme4.
The mcmcsamp function uses a new sampling method for the variance-covariance parameters that allows recovery from singularity. The update is not based on a sample from the Wishart distribution. It uses a redundant parameter representation and a linear least squares update.
CAUTION: Currently the results from mcmcsamp look peculiar and are probably incorrect. I hope it is just a matter of my omitting a scaling factor but I have seen patterns such as the parameter estimate for some variance-covariance parameters being the maximum value in the chain, which is highly unlikely.
The 'verbose' argument to lmer(), glmer() and nlmer() can be used instead of 'control = list(msVerbose = TRUE)'.