Needle in haystack testing

Repo for @hi_jamari's needle in haystack testing. Uses 200k tokens of transcripts from the acquired podcast episodes about Bershire Hathaway, Amazon, and Charlie munger. See haystack.txt

We'll be testing with various question/answer needle combos, see results below:

Test 1 Results

Question: "What color is Pongo's logo?" Needle: "Jamari: Pongo's logo is purple and white"

Llama3.1-405b (first 60k tokens only):

Mistral-large-2 (First 80k tokens only):