[sycl-free-inference-for-llms] Port and evaluate LLama3-8B and Granite-8B

[etiotto]

Meta has written the following PyTorch blog: https://pytorch.org/blog/cuda-free-inference-for-llms
They have evaluated Llama3-8B using Triton on A100 and H100 GPUs.
We should do the same for PVC after porting the code.

The instructions are as follows:

1 -Get the code:
git clone https://github.com/AdnanHoque/foundation-model-stack.git
git checkout amd_attn cd foundation-model-stack pip install -e. cd scripts/
2 - weights and tokenizer from: https://huggingface.co/meta-llama/Meta-Llama-3-8B/tree/main

3 - to run (update model path and tokenizer to your local drive) :
CUDA_LAUNCH_BLOCKING=1 CUDA_VISIBLE_DEVICES=0 python inference.py --architecture=llama --variant=3-8b --tokenizer="/net/storage149/autofs/css22/nmg/models/llama3-8b/base" --model_path="/net/storage149/autofs/css22/nmg/models/llama3-8b/base" --device_type cuda --model_source hf --compile
4 - script options are controlled in : https://github.com/.../blob/amd_attn/scripts/inference.py

[Dewei-Wang-sh]

depend on below issues to make it work.

• [FA2 performance] make sure end2end perf of FA2 match POC branch #1758
• [sycl-free-inference-for-llms] run llama3-8B with Pytorch for xpu and get the base line #2197
• [sycl-free-inference-for-llms] gemm kernel perf get up to ~290tflops #2194
• [sycl-free-inference-for-llms] flashattention kernel get up to ~100tflops #2195
• [sycl-free-inference-for-llms] integrate triton gemm/attention in pytorch for xpu #2196