Beam search: Truncated predictions while using length_penalty_weight

[shubhamagarwal92]

Hi

I am using Beam search for inference as described here.

The relevant 'model parameters' are:

inference.beam_search.beam_width: 10
inference.beam_search.length_penalty_weight: 1.0 

If I am not using the length_penalty_weight (second parameter above) while just specifying the beam_width (first parameter), the model is working fine with no truncations.

When I am using length_penalty_weight (with beam width 10), my predictions text file (predictions.txt as specified in Decoding with beam search ) is truncated while the dump numpy array (.npz file) contains the full length results.

Taking cue from model parameters specified here, I tried using the following parameters at inference:

inference.beam_search.beam_width: 10
inference.max_decode_length: 300	
inference.beam_search.length_penalty_weight: 1.0 

but getting the error "ValueError: inference.max_decode_length is not a valid model parameter"

1. Do, I have to specify any other model parameter (during training, I have used both max_decode_length:300 as well as target.max_seq_len:300)? It was strange, as the truncations are only happening when using the 'length_penalty_weight' (along with 'beam_width')

2. Should the results in 'predictions.txt' be the first (best/most probable) prediction if we produce the top-k predictions using the script in How to infer top-k output sequences? #158?

Thank you.