Biases

[Jorgeferrrus]

Dear Dr. Roth,

I am currently using your package to compute the unconditional and conditional biases after passing a pretest with different data. I understand that the unconditional bias is equivalent to "deltatrue." However, I am uncertain about the conditional bias. Should it be directly interpreted as "meanafterpretesting," or is there an additional calculation involved?

Thank you for your time.

Best regards,

Jorge

[jonathandroth]

If I understand your email correctly, I think the answer is yes. In the notation of the paper, `deltatrue' is \delta, and meanAfterPretesting is E[ \betahat | \betahat_{pre} \in B], where B is the set of "insignificant coefficients" Best, J

…

On Mon, Aug 12, 2024 at 5:46 AM Jorgeferrrus ***@***.***> wrote: Dear Dr. Roth, I am currently using your package to compute the unconditional and conditional biases after passing a pretest with different data. I understand that the unconditional bias is equivalent to "deltatrue." However, I am uncertain about the conditional bias. Should it be directly interpreted as "meanafterpretesting," or is there an additional calculation involved? Thank you for your time. Best regards, Jorge — Reply to this email directly, view it on GitHub <#7>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6EXFDUQFXXMHTS2SWLHWDZRCAAHAVCNFSM6AAAAABMLZ4FHSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ3DANJRGYZTOMY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[Jorgeferrrus]

Dear Dr Roth,

Thank you for your quick response!

I’m still having some trouble understanding the function's notation. In Figure 1 in the paper, the conditional bias is expressed as E[\tauhat − \tau^* | \betahat_{pre} \in B]. Based on my understanding, wouldn't the conditional bias in figure 1 be computed (with the results of the pretrends function) as:

\frac{1}{M} \sum meanAfterPretesting_post - \frac{1}{M} \sum \betahat_post ? (1)

I tried to replicate the results with one of the 12 papers you surveyed; however the result is much closer to the one in the paper if I don't substract \frac{1}{M} \sum \betahat_post to \frac{1}{M} \sum meanAfterPretesting_post, i.e. if I set the conditional bias equal to average of meanAfterPretesting_post. This makes me think that the approach (1) is wrong.

*** I refer to "meanAfterPretesting_post" and "\betahat_post" as the subvectors after treatment.

Best,

Jorge

[jonathandroth]

The conditional bias as you said is: E[\tauhat − \tau^* | \betahat_{pre} \in B] The pretrends function considers the case where \tau^* = 0 (everything comes from the pre-trend, so there is no treatment effect). So then the bias is just E[ \tauhat | \betahat \in B] = meanAfterPretesting.

…

On Mon, Aug 12, 2024 at 10:39 AM Jorgeferrrus ***@***.***> wrote: Dear Dr Roth, Thank you for your quick response! I’m still having some trouble understanding the function's notation. In Figure 1 in the paper, the conditional bias is expressed as E[\tauhat − \tau^* | \betahat_{pre} \in B]. Based on my understanding, wouldn't the conditional bias in figure 1 be computed (with the results of the pretrends function) as: \frac{1}{M} \sum meanAfterPretesting_post - \frac{1}{M} \sum \betahat_post ? *(1)* I tried to replicate the results with one of the 12 papers you surveyed; however the result is much closer to the one in the paper if I don't substract \frac{1}{M} \sum \betahat_post to \frac{1}{M} \sum meanAfterPretesting_post, i.e. if I set the conditional bias equal to average of meanAfterPretesting_post. This makes me think that the approach *(1)* is wrong. *** I refer to "meanAfterPretesting_post" and "\betahat_post" as the subvectors after treatment. Best, Jorge — Reply to this email directly, view it on GitHub <#7 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE6EXFHISBLLWJ2FGQMM7WDZRDCLHAVCNFSM6AAAAABMLZ4FHSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBUGE3DQNBQHE> . You are receiving this because you commented.Message ID: ***@***.***>