Cloud native Superclusters supercomputed neural network parallel(core/threads, TBB, vectorized instructions), heterogeneous(FPGA or GPU accelations) computed.
There's a lot of NN framework for parallel, heterogeneous computed on distributed computing. but this frameworks needed specific machine that vecdor dependent software. futher more, user frontend should be able to utilize a high-performance NN backend in a convenient web-based interface on any playform.
Also, we need container, k8s friendly cloud native framework.
Web application visualize your model graph and provide interface that communicate with neural network handle each layer or unit. system monitoring each computing nodes that have unit are also planned.
Scale out nerual network on cloud or on premise cluster based computing. loadbalance your trainning / test processes from big data input. so, this will reduce the NN's learning time. we can make decision agilely from insight we want by quickly being responsed from NN.
SYCL is an effective alternative. when we program the tensor, we don't need CUDA with bunch of codes in .cu and .cuh. otherwise,
we need type safe programming. so, no more .cl codes from OpenCL.
We must break free from lock-in to specific GPU vendors. High-performance and energy efficient machines can coexist and cooperative each other in distributed neural network.
Thus, we select Eigen tensor that have implemented SYCL backend for heterogeneous computing.
We need superfast, GPU accelated, and size optimized standard colmnar format.
We can maximize network communication performance in distributed neural network by using boost::beast framework based boost::asio async event driven programming.
To deal with future large distributed compile, use bazel fast build and unit test.