Is anyone finding the bug in embedding share? How to understand the design of tf.make_template's decorator?

[liyi193328]

Hi, smart guys:
I may find a bug in embedding share. Here is it. When setting embedding.share is True in config yaml.

Share source and target embedding the code in models/seq2seq_model.py lineth:126-150:

@property
@templatemethod("source_embedding")
def source_embedding(self):
  """Returns the embedding used for the source sequence.
  """
  return tf.get_variable(
      name="W",
      shape=[self.source_vocab_info.total_size, self.params["embedding.dim"]],
      initializer=tf.random_uniform_initializer(
          -self.params["embedding.init_scale"],
          self.params["embedding.init_scale"]))

@property
@templatemethod("target_embedding")
def target_embedding(self):
  """Returns the embedding used for the target sequence.
  """
  if self.params["embedding.share"]:
    return self.source_embedding
  return tf.get_variable(
      name="W",
      shape=[self.target_vocab_info.total_size, self.params["embedding.dim"]],
      initializer=tf.random_uniform_initializer(
          -self.params["embedding.init_scale"],
          self.params["embedding.init_scale"]))

It can't guarante source embedding and target embedding is the same, because these are in different name scopes (one is encode, one is decode). I find it when debuging model analysis .

Is there any nice way to solve it then .

And lastly, tf.make_template's decorator is used as often in the project. But I can't find any usage in code or function. Is any idea?