You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have a zero-masked bidirectional RNN for mini-batched data with variable lengths. I want to extract the last hidden output of the forward pass and the first hidden output of the backward pass as the representation of the sequence.
Since each sequence in the batch has different lengths, I can think of two possible ways to implement this:
i) Prepare two versions of the data, one with zeros padded on the left while the other with zeros padded on the right. The forward pass uses the left-padded data and use SelectTable(-1). Similarly, the backward pass uses the right-padded data and use SelectTable(1).
ii) Alternatively, I can implement a custom MaskZeroSelectTable(-1) to select the last hidden output that is not 0.
I guess this is not an uncommon usage, but I could not find an existing solution. Am I missing anything?
If there is no over-the-shelf implementation, what is the best way to implement this?
The text was updated successfully, but these errors were encountered:
ccsasuke
changed the title
Getting first and last hidden outputs for BiLSTM with variable-length batches
Getting first and last hidden outputs for BiLSTM with variable-length sequences
Dec 1, 2016
Hi @aron-bordin, it seems this didn't catch the attention of the devs.. I switched to some other framework for this function. Maybe you can try the workaround i) I mentioned?
Hi,
I have a zero-masked bidirectional RNN for mini-batched data with variable lengths. I want to extract the last hidden output of the forward pass and the first hidden output of the backward pass as the representation of the sequence.
Since each sequence in the batch has different lengths, I can think of two possible ways to implement this:
i) Prepare two versions of the data, one with zeros padded on the left while the other with zeros padded on the right. The forward pass uses the left-padded data and use SelectTable(-1). Similarly, the backward pass uses the right-padded data and use SelectTable(1).
ii) Alternatively, I can implement a custom MaskZeroSelectTable(-1) to select the last hidden output that is not 0.
I guess this is not an uncommon usage, but I could not find an existing solution. Am I missing anything?
If there is no over-the-shelf implementation, what is the best way to implement this?
The text was updated successfully, but these errors were encountered: