Sudoku-extreme dataset mismatch?

[sandorvasas]

Has anyone been able to reproduce anything around the mentioned 87% precision on sudoku-extreme?
There is apparently some confusion, at least for me. There are two kinds of sudoku-extreme datasets:

• this has 1k training samples: https://huggingface.co/datasets/sapientinc/sudoku-extreme-1k
• this has millions: https://huggingface.co/datasets/sapientinc/sudoku-extreme

Now the paper mentions the "Sudoku Extreme" dataset having 1K examples, but the code in the repository actually uses "sudoku-extreme" and not "sudoku-extreme-1k"!

I'm training my own implementation of the paper on sudoku-extreme-1k and the max cell accuracy I can reach is around 40%.

So is it indeed 87% accuracy on sudoku-extreme-1k, or the author made a mistake and in fact trained the model on the large sudoku-extreme dataset, and achieved 87% on that?

[sandorvasas]

nvm, I noticed the subsample_size argument for for setting up the sudoku dataset, so it picks N randomly from the full and generates it to the data folder.