Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate Weight initialization for LUTNets #8

Open
rdaly525 opened this issue Dec 22, 2017 · 0 comments
Open

Investigate Weight initialization for LUTNets #8

rdaly525 opened this issue Dec 22, 2017 · 0 comments

Comments

@rdaly525
Copy link
Owner

The stddev == sqrt(1/Fanin) comes from trying to keep the variance for each layer the same.
https://intoli.com/blog/neural-network-initialization/

I suspect that convnets are not training very well because I am doing naive weight initialization. I want to try to write down a similar formula for a LUTNet. to see if it can train better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant