Skip to content

tam.se wrong SE values for threshold parameters in PCM #30

@ghost

Description

Hi,

I've recently been using TAM to compute Partial Credit Model on simulated data (I've included a sample with 4 items and 1 group variable TT with this issue). I've been comparing the output to the results I get from stata's gsem command. I've found that the standard errors I get from TAM's tam.se don't match what I get from Stata, while everything else is fairly consistant. More specifically, when repeating over a large number of datasets, I've found that:

  • The SE reported by TAM for the group parameters beta has to be multiplied by almost exactly 1.4 to match Stata's (I've found the ratio is the same whatever configuration of simulated data I tried)
  • The SE for the threshold parameters of the items have, on average, to be multiplied by around 1.2 - 1.3 (this however does not appear to be a constant scaling factor, as it appears to vary between items and thresholds)

Replicate with given data

# import
dat <- read.csv("toy_4cat.csv")

# convert to long format, with each item being a separate observation of an individual
mod <- tam.mml(resp=dat[,c("item1","item2","item3","item4"],group=dat[,"TT"],irtmodel="PCM")

# Print SE
tam.se(mod)

TAM output

#### tam.se ####
$xsi
         item  N         est        se
1  item1_Cat1 NA -2.02809921 0.1842392
2  item1_Cat2 NA -0.72646573 0.1161076
3  item1_Cat3 NA  0.19718470 0.1034819
4  item2_Cat1 NA -1.18626278 0.1339661
5  item2_Cat2 NA -0.07457936 0.1083798
6  item2_Cat3 NA  0.74259121 0.1157891
7  item3_Cat1 NA -0.97391510 0.1296683
8  item3_Cat2 NA -0.22137412 0.1075579
9  item3_Cat3 NA  1.10198727 0.1204700
10 item4_Cat1 NA -0.08058468 0.1040038
11 item4_Cat2 NA  0.89309142 0.1161839
12 item4_Cat3 NA  1.96836950 0.1745778

$beta
  est.Dim1    se.Dim1
1 0.000000 0.00000000
2 0.152853 0.07138014

Stata output

---------------------------------------------------------------------------------------------------------------------
                                                                                                       <--95% IC -->
                          <---- DIF variables --->                                                     Lower   Upper
Items                                                  Threshold    Estimate    s.e.        z      p   Bound   Bound
---------------------------------------------------------------------------------------------------------------------
item1                                                          1       -2.03    0.21    -9.75  0.000   -2.43   -1.62
                                                               2       -0.73    0.14    -5.17  0.000   -1.00   -0.45
                                                               3        0.20    0.13     1.56  0.118   -0.05    0.45

item2                                                          1       -1.19    0.16    -7.53  0.000   -1.49   -0.88
                                                               2       -0.07    0.13    -0.56  0.576   -0.33    0.19
                                                               3        0.74    0.14     5.28  0.000    0.47    1.02

item3                                                          1       -0.97    0.15    -6.31  0.000   -1.28   -0.67
                                                               2       -0.22    0.13    -1.68  0.093   -0.48    0.04
                                                               3        1.10    0.14     7.64  0.000    0.82    1.39

item4                                                          1       -0.08    0.13    -0.64  0.525   -0.33    0.17
                                                               2        0.89    0.14     6.31  0.000    0.62    1.17
                                                               3        1.97    0.20     9.79  0.000    1.58    2.36

---------------------------------------------------------------------------------------------------------------------
Variance_T1                                                             1.13    0.13     8.65  0.000    0.87    1.38
TT                                                             1        0.15    0.10     1.51  0.132   -0.05    0.36
---------------------------------------------------------------------------------------------------------------------

I've not been able to pinpoint exactly what the cause might be. However, I've noticed in https://github.com/alexanderrobitzsch/TAM/blob/master/R/tam.se.R that tam.se appears to be using a quick SE computation method while the other option remains unimplemented. Is this "expected" behaviour for the quick SE method ?

Thanks in advance for your help !

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions