Rui/use system deps

[rmdg88]

• updated pyproject.toml to use pybind at build-system
• updated pyproject.toml include = [ ] to include more paths
• updated build.py
• add linux sdist build step
• updated cmake external deps to optionally use system deps and use pybind from poetry
• bumped external deps CXXOPTS to v3.2.0
• updated external deps LOGURU install method
• add rhel container to build with system deps
• add ci test step for windows and mac
• fix modules load_pretrained_nlp_models, load_training_data to properly construct URL and be compatible cross platform to load tests

[rmdg88]

regarding the sys deps build:

• the loguru-devel does not include loguru.cpp, it only include loguru.hpp i tried to update that in the existing files, but fails to work properly, i guess the code needs it (maybe the .rpm can be packaged with it) as workaround i've added the git clone and make install in the build_rhel.sh to move forward

• at the moment, there is only one test failing, train tokenizer this one:

  deepsearch-glm/tests/test_nlp.py

  Lines 693 to 723 in 6fa3bcb

  	# train tokenizer
  	def test_07B():
  	resources_dir = get_resources_dir()
  	txt_file = f"{resources_dir}/data/text/arxiv-abstracts-2020-Jan.txt"
  	assert os.path.exists(txt_file)
  	model_type = "unigram"
  	model_name = "test-tokenizer-model"
  	print(f"training on {txt_file}")
  	model_file = create_tok_model(
  	model_type=model_type, model_name=model_name, ifile=txt_file
  	)
  	assert os.path.exists(model_name + ".model")
  	assert os.path.exists(model_name + ".vocab")
  	# load custom tokenizer
  	def test_07C():
  	model_name = "test-tokenizer-model"
  	model_file = f"{model_name}.model"
  	assert os.path.exists(model_file)
  	model = init_nlp_model(
  	f"language;custom_spm({model_name}:{model_file})",
  	filters=["properties", "instances"],
  	)

see:

• dunno if is something related with sentencepiece_train we use 2 static libs in the compile from source version, or if is something else

• i've checked and the sentencepiece-devel installs both libs

• all the other tests pass in the sys deps version

[dolfim-ibm]

LGTM 💯