Skip to content

Commit

Permalink
Resolve #376
Browse files Browse the repository at this point in the history
The previous code is unfortunately passed a test case. When all sequences are shorter than maximum length, at 1st time step, the first dimension size of `self.noise` is 1 in TrimZero algorithm. Then, (Lazy) Dropout's `self.noise` is copied across time steps, presumably, by [this](https://github.com/Element-Research/rnn/blob/master/AbstractRecurrent.lua#L30), as a result, it can avoid an error `incorrect size: only supporting singleton expansion (size=1)` since the first dimension size of `self.noise` is always equal to 1.

Note that since Bayesian GRU with TrimZero should use monotonic sampling (the same dropout samplings across a batch) for dropouts, the performance is the same if an error is not occurred due to the distribution of sequence lengths.
  • Loading branch information
jnhwkim authored Jan 22, 2017
1 parent 2724eac commit e7f1ef2
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion Dropout.lua
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ function Dropout:updateOutput(input)
self.flag = false
end
if self.mono and self.noise:size(1) ~= input:size(1) then
self.noise = self.noise:expandAs(input)
self.noise = self.noise:narrow(1,1,1):expandAs(input)
end
self.output:cmul(self.noise)
elseif not self.v2 then
Expand Down

0 comments on commit e7f1ef2

Please sign in to comment.