update 2 #2
Open
update 2 #2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Fix grad conv im2col * Also fix depthwise * Enable prevously broken tests * Revert "Enable prevously broken tests" This reverts commit d648fdd. * Add explicit im2col test * Fix and test third case * More tests now pass
* unbreak gpu conv test * cleanup
* Add EnzymeRule for conv * Fix * Add missing file * Change to enzymecore ext * attempt fix * Finish enzymecore rewrite * Add missing file * Also add gather * Additional functions, tests, and fixes * minor fixup * Add pooling * Add dropout * Fix scatter bug * fix pool * More fixups * fix up pool * split conv/depth * Cleanup rule * Fix minor test bug * Fix depthwise conv * Fix typo * Bound tests * Failing to extension * Add file * Address review * Remove inlining
* Fix typo in version comparison * Allow non-Int64 indices in scatter * Disable Enzyme for AMDGPU * Refactor
… existing compat) (#549) Co-authored-by: CompatHelper Julia <[email protected]>
* AbstractGPUArray -> AnyGPUArray * tests * don't test Enzyme * add test on discontinuous view * Update test/runtests.jl
* Set compatibilities for standard packages * Update Project.toml * Update Project.toml * Update Project.toml * Update Project.toml * Uncomment Random --------- Co-authored-by: Anton Smirnov <[email protected]>
* Bump AMDGPU compat to 0.8 * Bump AMDGPU CI to 1.10
Co-authored-by: CompatHelper Julia <[email protected]>
* Use julia-actions/cache This does everything automatically for us and should speed up CI times.
* Add dependencies * Add code * Add test * clear docs * Fix FiniteDifferences.fdm_central * Include Refactoring * Code cleaning and fix adjoint for trivial rotations * Code cleaning * Fix bug with even and odd arrays and trivial rotations * First parts of test are generalized, not gradients yet * Add gradtests, some fail * Tests working and subtle bug fixed for trivial rotations * Fix space before && [skip ci] * Add documentation and fix issue with FillArrays * Fix function name * Fix size(arr) * Relax some tests since they failed on CUDA * Test with rel error of 1f-2 * Refine rotation tests * Introduce show statement for buildkite * Remove show statement, introduce even test case again * Rename midpoint to rotation_center and change rounding * Add more tests, nearest neighbour fails sometimes * Lower tolerance tests * Proper error handling * Add underscore _ to internal methods. Clean docs * Change to Float64 test * Lower testing accuracy for Float64 * Revert "Lower testing accuracy for Float64" This reverts commit fcba1c3. * Rerun CI * Fix typo, rerun CI * Improve docstring [skip ci]
In test mode, the CUDA cuDNN implementation of batchnorm was not matching the CPU batchnorm in FLUX. In FLUX, with track_stats=False, the mean and variance of the current batch are used. Here, mean and variance were initialized to 0 and 1, respectively, and passed to cudnnBatchNormalizationForwardInference. To fix this, we need to calculate the mean and variance over the current batch to match the CPU implementation. Unfortunately, cudnnBatchNormalizationForwardInference requires a trained running mean and variance. However, batchnorm train and test should be identical without tracked stats since they both normalize over the current batch. As a result we can use cudnnBatchNormalizationForwardTraining in test mode as well, which works without a running mean and variance.
remove `NNPACK` and move `ForwardDiff` to an extension
* Bump patch version to release new Enzyme support * chore: force latest Enzyme install [DON'T MERGE] * revert: "chore: force latest Enzyme install [DON'T MERGE]" This reverts commit 16f8075. --------- Co-authored-by: Avik Pal <[email protected]>
* Fix backtick * Update language highlight tags for fenced code samples * Update whitespace * Use TeX primitives * Update reference links * Remove duplicate backticks * Fix admonition block * Add backticks
- Use `1.10` in Buildkite CI (and `nightly` for CUDA). - Use `lts`, `1` and `pre` in GitHub CI. - Add compat GPUArraysCore for `0.2`. - Bump ChainRulesCore to `0.25`.
* Add SpecialFunctions as dependency * add full `gelu`, changes old `gelu` -> `gelu_fast` * Add tests and docs * Change names: `gelu_fast` -> `gelu`, `gelu` -> `gelu_full` * Rename gelus and use OpenLibm_jll instead of SpecialFunctions.jl for erf * Create NNlibSpecialFunctionsExt for gelu_erf * specific import in SpecialFunctions extension * add gelu export to list of aliases
We mistakenly did not register v0.9.28.
… at version 2, (keep existing compat)
…01-01-26-30-687-03248299932 CompatHelper: add new compat entry for SpecialFunctions in [weakdeps] at version 2, (keep existing compat)
This should fix #631.
Add type alias for gelu -> gelu_tanh
#633) * don't spawn if only one job * fix what appears to be a typo * revertme: temporary test * fix check * rm test * add NNlib.ALLOW_THREADING control * use ScopedValues * use `@with` to avoid new scope * rename do_work functions * add note * v0.9.29
…xisting compat) (#637) Co-authored-by: CompatHelper Julia <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.