Skip to content

Commit 17837ef

Browse files
committed
bug #822 [Demo] Use TextSplitter for indexing in demo due to long blog posts (chr-hertel)
This PR was merged into the main branch. Discussion ---------- [Demo] Use TextSplitter for indexing in demo due to long blog posts | Q | A | ------------- | --- | Bug fix? | yes | New feature? | no | Docs? | no | Issues | | License | MIT Currently the demo fails to index the Symfony Blog since there are longer blog posts that are too long for the token limit of the embeddings endpoint - using the splitter solves that. 🥳 Commits ------- 9f86de7 Use TextSplitter for indexing in demo due to long blog posts
2 parents 5db564b + 9f86de7 commit 17837ef

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

demo/config/packages/ai.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ ai:
7070
filters:
7171
- 'app.filter.week_of_symfony'
7272
transformers:
73+
- 'Symfony\AI\Store\Document\Transformer\TextSplitTransformer'
7374
- 'Symfony\AI\Store\Document\Transformer\TextTrimTransformer'
7475
vectorizer: 'ai.vectorizer.openai'
7576
store: 'ai.store.chroma_db.symfonycon'
@@ -89,6 +90,7 @@ services:
8990
$store: '@ai.store.chroma_db.symfonycon'
9091

9192
Symfony\AI\Store\Document\Loader\RssFeedLoader: ~
93+
Symfony\AI\Store\Document\Transformer\TextSplitTransformer: ~
9294
Symfony\AI\Store\Document\Transformer\TextTrimTransformer: ~
9395

9496
app.filter.week_of_symfony:

0 commit comments

Comments
 (0)