From reliability (CTT) to test information (IRT) and back

Test information in item response theory (IRT) is defined as:
 




The denominator is basically the squared value of the standard error of measurement (SEM). The SEM is defined in classical test theory (CTT) as:

In IRT, we assume T to have a variance of 1 in the population, so let's say Sxx = 1. (Although the variance of theta is not necessarily the same as the variance of the observed test scores. I just do not know what the observed version of theta would be in IRT.). Then:

It follows that:

and

Calculating reliabilities in R

The psych package in R provides a function for estimating McDonald's (1999) \omega. However, one has to specify the number of factors to extract, while in many cases one may want to calculate \omega_{h} for a unidimensional solution. Furthermore, the function requires a correlation matrix or raw data. Sometimes you have a covariance matrix, for example if you use an EM algorithm (implemented in the norm package in R) for estimation of first and second order moments (means and (co)variances). The following code easily allows you to compute testscore variance and Cronbach's alpha in the process, as well.


R code

# The following code is optional, provided to make it a working example:
# covs <- c(
# 0.64127029, 0.1483167, 0.1759022, 0.16268973, 0.15268583, 0.15584803, 0.09879029,
# 0.14831673, 0.7555890, 0.3348555, 0.10005077, 0.19924240, 0.17755548, 0.17137305,
# 0.17590216, 0.3348555, 0.8843859, 0.12027226, 0.35636799, 0.18662961, 0.27231721,
# 0.16268973, 0.1000508, 0.1202723, 0.54879992, 0.04977889, 0.09217347, 0.01885471,
# 0.15268583, 0.1992424, 0.3563680, 0.04977889, 0.84438811, 0.33348737, 0.21997734,
# 0.15584803, 0.1775555, 0.1866296, 0.09217347, 0.33348737, 1.50964839, 0.14793777,
# 0.09879029, 0.1713731, 0.2723172, 0.01885471, 0.21997734, 0.14793777, 0.53258872)
# covmat <- matrix(covs,7,7)

library(psych)
Vx <- sum(covmat) # covmat is a square (co)variance matrix, Vx is the testscore variance
sumvar <- tr(covmat) # sumvar is the sum of all item variances
var <- diag(covmat) # var is a vector with item variances
sumcov <- Vx - sumvar # sumcov is the sum of all item covariances
n <- 7
alpha <- n/(n-1) * sumcov / Vx # alpha is Cronbachs alpha
commu <- principal(covmat, rotate="none")$communality # commu is a vector with all item communalities
uniqvar <- var*(1-commu) # uniqvar is a vector with all unique variances
omega <- 1 - (sum(uniqvar)/Vx) # omega provides an estimate of the proportion of testscore variance attributable to a single underlying factor


References

Cronbach L J. Coefficient Alpha and the internal structure of tests. Psychometrika 16:297-334, 1951.

McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, N.J.: Erlbaum Associates.

Revelle, W., & Zinbarg, R. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154.

Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbachs alpha. Psychometrika, 74 (1), 107-120.