Investigate DataCurate4LLMs repository for inclusion as a dependency of SDG

[bbrowning]

Investigate the repository at https://github.com/krishnatejakk/DataCurate4LLMs to see if it's suitable to include as a dependency of InstructLab/SDG. Is the license compatible? Is that project maintained? What's it's maturity level as far as releases, automated testing? Are we prepared to contribute fixes or code to that repo as necessary and work with the maintainer(s) to ensure they get merged?

From a purely technical perspective, do we have any conflicting dependencies? What are the hardware expectations, embedding models required, etc for us to be able to run this within SDG? Does it add any new requirements for our users and their systems?