Skip to content

Conversation

@mineichen
Copy link
Contributor

  • Remove tokio/async_stream/tokio_stream dependencies and use lighter futures_util/pin-project-lite instead
  • Unify streaming apis with a Lines adapter
    • Most of them had problems, if one Bytes would contain multiple lines, which could happen everywhere
  • use try_next() instead of .next() to improve readability
  • Share common code in streaming and non-streaming methods

Breaking:

  • Change ChatMessageResponse to return OllamaError instead of ()
  • Remove GenerationResponseStreamChunks and stream all items independently
  • Chat returns Error if one occurs... The Caller can still filter errors out and continue
  • Non-Streaming consistenly returns Reqwest-Error instead of OllamaError::Other if error-text cannot be gathered from response (sometimes it was res.text().await?, sometimes res.text().await.unwrap_or_else(|e| e.to_string()),)

Functionaity should stay almost the same. Sorry for the big chunk... What i'd like to send separately later:

  • Allow WASM-Build (currently reqwest::bytes_stream() is !Send)
  • Remove all stream properties from the requests and send them with WithSendField-heper
  • Introdce pub(crate) Ollama::send_request() to remove duplication of building request with common stuff

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant