SwiGLU MLP: parameter-neutral gated activation over LeakyReLU^2#676

they-call-me-god wants to merge 1 commit intoopenai:mainopenai/parameter-golf:mainfrom

they-call-me-god:swiglu-submissionthey-call-me-god/parameter-golf:swiglu-submissionCopy head branch name to clipboard