Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apparent discrepency between code and documentation for fractional bias and fractional error #173

Open
pavlovc2 opened this issue Jan 4, 2024 · 2 comments

Comments

@pavlovc2
Copy link

pavlovc2 commented Jan 4, 2024

The documentation states that the numerator and denominator should be summed separately to calculate fractional bias and error.
image

However, the code for DomainStats in AQ_Misc_Functions.R appears to calculate these metrics as the sum after the fraction is calculated for each observation/model pair. The code also multiplies the result by a 100 to report as a percentage rather than a fraction.

https://github.com/USEPA/AMET/blob/1.5/R_analysis_code/AQ_Misc_Functions.R#L652

fb <- signif(c(fb,(sum((data.df$mod_val-data.df$ob_val)/((data.df$mod_val+data.df$ob_val)/2)))/length(data.df$stat_id))*100,3)
fe <- signif(c(fe,(sum(abs(data.df$mod_val-data.df$ob_val)/((data.df$mod_val+data.df$ob_val/2)))/length(data.df$stat_id))*100,3)

Am I correct that this is a discrepancy, and can this be updated to be consistent with the correct approach?

@wkappel
Copy link
Collaborator

wkappel commented Jan 4, 2024

Nathan,

Thanks for your comment. It turns out the definition in the document is the one that is incorrect. The summing should be taking place outside of the brackets (as is done in the actual AMET R code). So, we will note this as a correction needed to this document.

Let me know if you have any further questions. Thanks!

Wyat

@pavlovc2
Copy link
Author

pavlovc2 commented Jan 4, 2024

Got it, thank you for the quick clarification!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants