Open
Description
I am trying to use Sambanova API which is compatible with OpenAI SDK. Would love to see how fast inference can benefit here and also use open source LLMs.
Below is an example of Sambanova API call:
import os
import openai
client = openai.OpenAI(
api_key=os.environ.get("SAMBANOVA_API_KEY"),
base_url="https://api.sambanova.ai/v1",
)
response = client.chat.completions.create(
model="Meta-Llama-3.1-8B-Instruct",
messages=[{"role":"system","content":"You are a helpful assistant"},{"role":"user","content":"Hello"}],
temperature=0.1,
top_p=0.1
)
print(response.choices[0].message.content)Metadata
Metadata
Assignees
Labels
No labels