Skip to content

Commit e7f1ef2

Browse files
authored
Resolve #376
The previous code is unfortunately passed a test case. When all sequences are shorter than maximum length, at 1st time step, the first dimension size of `self.noise` is 1 in TrimZero algorithm. Then, (Lazy) Dropout's `self.noise` is copied across time steps, presumably, by [this](https://github.com/Element-Research/rnn/blob/master/AbstractRecurrent.lua#L30), as a result, it can avoid an error `incorrect size: only supporting singleton expansion (size=1)` since the first dimension size of `self.noise` is always equal to 1. Note that since Bayesian GRU with TrimZero should use monotonic sampling (the same dropout samplings across a batch) for dropouts, the performance is the same if an error is not occurred due to the distribution of sequence lengths.
1 parent 2724eac commit e7f1ef2

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

Dropout.lua

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ function Dropout:updateOutput(input)
4444
self.flag = false
4545
end
4646
if self.mono and self.noise:size(1) ~= input:size(1) then
47-
self.noise = self.noise:expandAs(input)
47+
self.noise = self.noise:narrow(1,1,1):expandAs(input)
4848
end
4949
self.output:cmul(self.noise)
5050
elseif not self.v2 then

0 commit comments

Comments
 (0)