As a methodologist for the social sciences, I’ve spent years helping people see their research forest for the trees — or rather, their research question for the data.
But I also enjoy thinking along about life’s small dilemmas: (inter)personal challenges, the weird world that is academia, or how to design your shady backyard. I have some experience in these areas as a regular person, PhD candidate mentor, faculty council representative, and notorious hobby collector.
On this page I answer questions big and small, statistics to sketchbooks, wherever curiosity wanders.
The rules are simple: You ask your question and let me know if you’d like your name included when I post it here. I’ll choose which questions I answer, at whatever pace suits me. I won’t weigh in on medical issues (I’m not that kind of doctor) or politics — but I might expand on your intensive longitudinal data...
or what pillow to pick for your couch, for example.
Ask Noémi
Do you need formal or fast help with your research?
Contact me at work or request a consultation at Utrecht University's Methodology & Statistics department.
Sandro Menegola asks:
[...] In the videolecture about "measurement modeling & reliability with the MEAR model", you mention that if occasion-specific fluctuations are true, it is reasonable that they fluctuate across variables.
I was wondering whether in a "multivariate" MEAR(1), it would be possible to better distinguish measurement error from true occasion-specific fluctuations by modeling the correlations between occasion-specific fluctuations across variables (at one time point). I would assume that the correlations would capture at least some of the true variance, while the remaining would be a more precise estimate of the measurement error.
Interesting! I already did estimate those covariances among the time-point specific residuals in multivariate versions of this model, the MEVAR model. And I generally make the point there that one wouldn't expect measurement errors to be correlated across variables, so when we see those correlations - and we do - this implies that the time-point specific errors capture something else next to measurement error variance. But I haven't so far actively incorporated that in the reliability estimate; That is, to for example add that 'proportion of shared variance' among the time point specific errors to the reliability estimate, which would make sense. I will give that some thought.
Note that this is the basic idea in internal consistency approaches, where people use a factor structure to account for measurement error, and the shared variance among those items is considered part of the true score variance (the reliable part). But then the idea is that you specify items that all measure the same underlying construct, which is typically formalized with a latent variable model.
My current thoughts about taking those occasion-specific covariances from the MEVAR model into account for the reliability estimates, is simply to calculate the the amount of unshared-variance remaining in those time-point specific errors (e.g., using the proportion of shared variance), and to offset that against the total within-person variance to get the proportion of measurement error variance. The reliability would then be 1 minus that proportion. In the case that we have more than two variables, resulting in multiple of these covariances, I would suggest just taking the largest correlation for the variable of interest (the largest amount of shared variance) to update its reliability estimate.
In the multilevel model (in the n=1 model this would happen by design) it is important to do that per person, and I also think it would make most sense to ensure the covariance matrices can differ for each person, that is, have random covariance matrices in the multilevel model, as I do in my 2019 paper (see refs below or the link above).
Using the Bayesian samples for each person this would be straightforward to calculate (and get credible intervals for it, etc). However, if using DSEM in Mplus it is terribly difficult to obtain those samples per person (due to their software development choices), so I would recommend using JAGS (or STAN) instead where they are easily obtained. IJsbrand Leertouwer specifies code to fit (multilevel) MEAR models and to obtain reliability estimates from them in R and JAGS in a recent manuscript , although we mainly use univariate models, because these generally have much less convergence issues. Maybe we can update it to include these ideas at some point!
Refs:
Leertouwer, IJ., Keijsers, L., van Roekel, E., & Schuurman, N. K. (2025, June 18). A Practical Guide to Estimating Reliability of Intensive Longitudinal Data. https://doi.org/10.31234/osf.io/uq4sk_v1
Schuurman, N. K., & Hamaker, E. L. (2019). Measurement error and person-specific reliability in multilevel autoregressive modeling. Psychological methods, 24(1), 70.
Frederik Dornonville de la Cour asks:
[...] I am an applied researcher in brain injury rehabilitation with a neuropsychological background, currently working on an EMA/ESM study exploring moment-to-moment experiences of fatigue. In this study, 17 participants rated their fatigue on 0-10 scale, 10 times/day over 7 consecutive days.
I recently read your excellent preprint with Leertouwer as first author, which provides a practical guide to estimating within-person reliability using MEAR models. I have downloaded the R code and successfully applied it to my dataset.
My question concerns how to interpret the reliability estimate in terms of the original scale. Specifically, how it might relate to the standard error of measurement (SEM). For example, after running the ML MEAR model, I obtained a within-person reliability estimate of 0.725. I am curious how this translates into expected measurement error. If a participant reports a fatigue level of 7 at a given moment, how much error should I expect, and where might the true score plausibly lie on the scale? [...]
The person and item-specific reliability estimate(s) from the MEAR models are in this sense much like other reliability estimates: It is the estimated proportion of true-score variance out of the total variance of the variable you are validating. This implies that the estimated proportion of error variance out of the total variance is 1 minus that reliability. If you want to go from the error variance to the standard error of measurement, you take the square root of the variance. So, SEM = sqrt(total_variance x (1-R)).
This is the same as the more commonly presented equation that uses the total standard deviation instead of the total variance: SEM = total_sd x sqrt(1-R).
So in order to interpret the reliability coefficient in terms of the original scale, you need to know the total variance or standard deviation of the scale. Or, the estimated error variance of the scale (of which you can then take the square root). The error variance is estimated as part of the MEAR model, conveniently.
If you extract the Bayesian samples (iterations) for the error variance from the model, and take the square root from each of those samples, and then take the mean or median and appropriate quantiles from those samples you get the Bayesian point estimate and credible interval for the standard error of measurement. Those Bayesian samples should be automatically stored for you in a folder after you have run the analyses of IJsbrand, so in essence you should have everything you need to obtain the SEM.
You can use the SEM as usual to get an impression of the intervals around a true score (using z-scores), given that we assume normally distributed errors in the MEAR model. Keep in mind though that there is uncertainty around the SEM itself as well, so there is also uncertainty around such an interval. And keep in mind that for our MEAR approach, the interval and its uncertainty will differ for each participant and variable, just like the SEM, measurement error variance, and reliability coefficients.
If you are unsure how to work with the Bayesian samples, let me know and I can give you some pointers. And maybe we'll add the SEM to the standard results in the future, that could be nice.
Danielle McCool asks:
Should I paint my ceiling (in a different color than white)?
Yes, you should paint your ceiling in a different color! I think that if you feel the slightest inclination to paint a ceiling non-white it should be done. I think painting ceilings white is one of those habits people started to treat like a rule. And then they may even hint that you are being a little bit crazy for thinking of violating that rule. But it is not a rule, its a (potentially bad) habit, and you can do whatever the heck you want in your own home!
But aside from you being allowed to do as you please here, is it also a good idea? I think it often is, but it of course depends on the exact circumstances (and a matter of personal taste clearly). In Danielle's case we are talking about a hallway that has a vibrantly colored wallpaper with birds and flowers on it, and a dark purple background. Keeping the ceilings white creates, in my opinion, a too stark contrast with the relatively dark walls. This could make the hallway look harsh and uninviting, and the wallpaper more 'kitsch' than desired. I recommend picking one of the colors on the wallpaper and using that (or a variant of it) as the color on the ceiling. My favorite option in Danielle's case was a soft light purple-pink (After she asked this question Danielle and I spent too much time trying out various colors using a picture of her hallway). Danielle in the end decided on a nice dark blue and I'm very curious to see the result! As a sidenote: If you need inspiration to go full color - like purple walls and pink ceilings - check out the Dutch make-over show 'Jelka op eigen houtje'.
Now readers may have various follow-up questions, like "Should I always paint my ceiling a different color than white" or "Should I now feel bad that I didn't paint my ceiling in a different color?" or " Wait a minute Noémi, are YOUR ceilings white?".
The answer to the first two questions are, no, it can be a good idea to keep your ceilings white. It can make the room feel more spacious and fresh especially if your ceilings aren't very tall. It can brighten a room because it reflects more light, and optically 'lift' the room by being lighter than the surroundings. If your ceilings are very tall however it can be a good idea to paint your ceilings non-white for the same reason, maybe even painting a top strip of the walls along with it, to make the room feel more cozy. You can also paint a part your ceilings to create a little room inside a room, much like people use rugs on their floors. You can also do this with wallpaper on your ceiling. And lets not forget the option in general to go for murals on your ceiling (I want to do this at some point somewhere in my house). In my own home some of the ceilings are white, in one room it is a gray-white, in two others it is a light bluegreengrey. In my house I did opt for light colors and those who aren't white look white, but they are not (and white would look harsher).
In any case, if you feel inclined to paint your ceilings it is probably because something is not quite right with keeping them white (like a too harsh contrast). So I say go for it and paint it.
