feat(s3cpp): add streaming file uploads and configurable endpoints

README.md

@@ -193,3 +193,7 @@ $ docker run -d -p 9000:9000 -p 9001:9001 \
 ```
 
 The full test suite contains 63 tests
+
+## Architecture Decisions
+
+- [ADR 0001: Streaming File Uploads and Configurable S3 Endpoints](docs/adr/0001-streaming-file-upload-and-custom-endpoints.md)

docs/adr/0001-streaming-file-upload-and-custom-endpoints.md

@@ -0,0 +1,96 @@
+# ADR 0001: Streaming File Uploads and Configurable S3 Endpoints
+
+- Status: Accepted
+- Date: 2026-06-12
+
+## Context
+
+`s3cpp` supports uploading an in-memory string, but it does not provide a
+file-backed upload operation. Supporting file uploads without loading the
+complete file into memory requires:
+
+- upload progress reporting and cancellation;
+- HTTP and HTTPS custom endpoints;
+- a configurable AWS signing region;
+- `Content-Type` and `Content-Length` metadata on uploaded objects.
+
+The existing `PutObject` implementation accepts an in-memory string. This is
+appropriate for small objects, but it causes memory usage to grow with the file
+size and cannot report streaming upload progress.
+
+AWS Signature Version 4 normally includes the SHA-256 hash of the complete
+payload. A streamed file therefore needs its hash before the signed HTTP
+request starts.
+
+## Decision
+
+Add a file-backed single-request `PutObjectFile` operation.
+
+`PutObjectFile` performs these steps:
+
+1. Read the file and calculate its SHA-256 hash with OpenSSL EVP.
+2. Sign the request using the precomputed payload hash.
+3. Stream the file through libcurl using `CURLOPT_UPLOAD`.
+4. Report progress through libcurl's transfer progress callback.
+
+The callback returns `true` to continue or `false` to cancel the transfer.
+
+Add `HttpFileRequest` as a separate request type instead of adding file state
+to `HttpBodyRequest`. This keeps in-memory and file-backed uploads explicit and
+prevents accidental buffering of file contents.
+
+Extend the custom-endpoint `S3Client` constructor with:
+
+- `useHttps`, controlling the URL scheme;
+- `region`, used by AWS Signature Version 4.
+
+Keep the existing constructor as a compatibility overload. It defaults to HTTP
+and the `us-east-1` region, matching the previous custom-endpoint behavior.
+
+Apply optional `Content-Type` and `Content-Length` values in the existing
+in-memory `PutObject` implementation as well.
+
+## Consequences
+
+### Positive
+
+- File memory usage remains bounded regardless of object size.
+- Applications can display upload progress and cancel an upload.
+- The same client can connect to HTTP MinIO instances and HTTPS S3-compatible
+  services.
+- File uploads are signed with the actual payload hash.
+- Existing users of the custom-endpoint constructor remain source compatible.
+
+### Negative
+
+- Each file is read twice: once for SHA-256 calculation and once for upload.
+- Upload begins only after the initial hashing pass completes.
+- The implementation uses a single S3 `PutObject` request and does not support
+  multipart upload, retrying individual parts, or resuming an interrupted
+  upload.
+- A single `PutObject` is limited to 5 GiB by the S3 API.
+- The new implementation adds direct OpenSSL EVP and filesystem usage.
+
+## Alternatives Considered
+
+### Load the complete file into a string
+
+Rejected because memory consumption would scale with file size and progress
+would only describe sending an already-buffered payload.
+
+### Use unsigned payload mode
+
+Rejected as the default because support differs between S3-compatible servers
+and it weakens payload integrity guarantees.
+
+### Implement multipart upload
+
+Deferred. Multipart upload is the appropriate solution for large files,
+per-part retries, and resumable transfers, but requires additional S3
+operations and lifecycle handling. It should be introduced in a separate ADR.
+
+## Scope
+
+This decision covers single-request file uploads and endpoint configuration.
+It does not define multipart upload, download streaming, retry policy, or
+persistent transfer state.

src/s3cpp/auth.cpp

@@ -29,8 +29,16 @@ template <typename T> void AWSSigV4Signer::sign(HttpRequestBase<T> &request) {
         "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855";
   }
   request.header("x-amz-content-sha256", payload_hash);
+  sign(request, payload_hash);
+}
+
+template <typename T>
+void AWSSigV4Signer::sign(HttpRequestBase<T> &request,
+                          const std::string &payload_hash) {
+  request.header("x-amz-content-sha256", payload_hash);
 
   // Skip signing for anonymous requests
+  const bool is_anonymous = access_key.empty() || secret_key.empty();
   if (is_anonymous) {
     return;
   }
@@ -87,17 +95,18 @@ AWSSigV4Signer::createCannonicalRequest(HttpRequestBase<T> &request,
   std::string url = request.getURL();
 
   // URI
-  std::string uri{};
-  if (size_t bpos = url.find("amazonaws.com"); bpos != std::string::npos) {
-    uri = url.erase(0, bpos + 13);
+  const size_t scheme_end = url.find("://");
+  const size_t authority_start =
+      scheme_end == std::string::npos ? 0 : scheme_end + 3;
+  const size_t path_start = url.find_first_of("/?", authority_start);
+
+  std::string uri;
+  if (path_start == std::string::npos) {
+    uri = "/";
+  } else if (url[path_start] == '?') {
+    uri = "/" + url.substr(path_start);
   } else {
-    // Assume localhost:XXXX (dirty, sorry :( i know)
-    size_t path_start = url.find('/', 7);
-    if (path_start != std::string::npos) {
-      uri = url.substr(path_start);
-    } else {
-      uri = "/";
-    }
+    uri = url.substr(path_start);
   }
   size_t begin_q = uri.find("?");
   const std::string cannonical_uri =
@@ -227,9 +236,14 @@ AWSSigV4Signer::deriveSigningKey(const std::string request_date) {
 template void AWSSigV4Signer::sign<HttpRequest>(HttpRequestBase<HttpRequest> &);
 template void
 AWSSigV4Signer::sign<HttpBodyRequest>(HttpRequestBase<HttpBodyRequest> &);
+template void
+AWSSigV4Signer::sign<HttpFileRequest>(HttpRequestBase<HttpFileRequest> &,
+                                      const std::string &);
 template std::string AWSSigV4Signer::createCannonicalRequest<HttpRequest>(
     HttpRequestBase<HttpRequest> &, const std::string &);
 template std::string AWSSigV4Signer::createCannonicalRequest<HttpBodyRequest>(
     HttpRequestBase<HttpBodyRequest> &, const std::string &);
+template std::string AWSSigV4Signer::createCannonicalRequest<HttpFileRequest>(
+    HttpRequestBase<HttpFileRequest> &, const std::string &);
 
 } // namespace s3cpp

src/s3cpp/auth.h

@@ -17,6 +17,8 @@ class AWSSigV4Signer {
         secret_key(std::move(secret)) {}
 
   template <typename T> void sign(HttpRequestBase<T> &request);
+  template <typename T>
+  void sign(HttpRequestBase<T> &request, const std::string &payload_hash);
 
   template <typename T>
   std::string createCannonicalRequest(

src/s3cpp/httpclient.cpp

@@ -2,6 +2,7 @@
 #include <curl/easy.h>
 #include <expected>
 #include <format>
+#include <cstdio>
 #include <s3cpp/httpclient.h>
 #include <stdexcept>
 #include <string>
@@ -32,6 +33,10 @@ std::expected<HttpResponse, std::string> HttpBodyRequest::execute() {
   }
 }
 
+std::expected<HttpResponse, std::string> HttpFileRequest::execute() {
+  return client_.execute_upload(*this);
+}
+
 std::expected<HttpResponse, std::string>
 HttpClient::execute_get(HttpRequest &request) {
   if (!curl_handle) {
@@ -193,6 +198,68 @@ HttpClient::execute_post(HttpBodyRequest &request) {
                       std::move(headers_buf));
 }
 
+std::expected<HttpResponse, std::string>
+HttpClient::execute_upload(HttpFileRequest &request) {
+  if (!curl_handle) {
+    return std::unexpected<std::string>("cURL handle is invalid");
+  }
+
+  FILE *file = std::fopen(request.getFilename().c_str(), "rb");
+  if (!file) {
+    return std::unexpected<std::string>(
+        std::format("unable to open file: {}", request.getFilename()));
+  }
+
+  std::string body_buf;
+  std::map<std::string, std::string, LowerCaseCompare> headers_buf;
+  auto headers = request.getHeaders();
+  headers.insert(this->getHeaders().begin(), this->getHeaders().end());
+  struct curl_slist *list = nullptr;
+  for (const auto &[key, value] : headers) {
+    list = curl_slist_append(list, std::format("{}: {}", key, value).c_str());
+  }
+
+  curl_easy_reset(curl_handle);
+  curl_easy_setopt(curl_handle, CURLOPT_URL, request.getURL().c_str());
+  curl_easy_setopt(curl_handle, CURLOPT_UPLOAD, 1L);
+  curl_easy_setopt(curl_handle, CURLOPT_READDATA, file);
+  curl_easy_setopt(curl_handle, CURLOPT_INFILESIZE_LARGE, request.getFileSize());
+  curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_callback);
+  curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, &body_buf);
+  curl_easy_setopt(curl_handle, CURLOPT_HEADERFUNCTION, header_callback);
+  curl_easy_setopt(curl_handle, CURLOPT_HEADERDATA, &headers_buf);
+  curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, list);
+  curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, request.getTimeout());
+
+  const auto &progress = request.getProgressCallback();
+  if (progress) {
+    curl_easy_setopt(curl_handle, CURLOPT_NOPROGRESS, 0L);
+    curl_easy_setopt(
+        curl_handle,
+        CURLOPT_XFERINFOFUNCTION,
+        +[](void *userdata, curl_off_t, curl_off_t, curl_off_t upload_total,
+            curl_off_t uploaded) -> int {
+          const auto *callback =
+              static_cast<const UploadProgressCallback *>(userdata);
+          return (*callback)(uploaded, upload_total) ? 0 : 1;
+        });
+    curl_easy_setopt(curl_handle, CURLOPT_XFERINFODATA, &progress);
+  }
+
+  const CURLcode code = curl_easy_perform(curl_handle);
+  long response_code = 0;
+  curl_easy_getinfo(curl_handle, CURLINFO_HTTP_CODE, &response_code);
+  curl_slist_free_all(list);
+  std::fclose(file);
+
+  if (code != CURLE_OK) {
+    return std::unexpected<std::string>(
+        std::format("libcurl upload error: {}", curl_easy_strerror(code)));
+  }
+  return HttpResponse(static_cast<int>(response_code), std::move(body_buf),
+                      std::move(headers_buf));
+}
+
 std::expected<HttpResponse, std::string>
 HttpClient::execute_delete(HttpBodyRequest &request) {
   if (!curl_handle) {

src/s3cpp/httpclient.h

@@ -6,6 +6,7 @@
 #include <curl/curl.h>
 #include <curl/easy.h>
 #include <expected>
+#include <functional>
 #include <map>
 #include <stdexcept>
 #include <string>
@@ -17,6 +18,9 @@ class HttpClient;
 
 enum class HttpMethod { Get, Post, Put, Head, Delete };
 
+using UploadProgressCallback =
+    std::function<bool(curl_off_t uploaded, curl_off_t total)>;
+
 struct LowerCaseCompare { // A custom lambda to sort keys alphabetically
   bool operator()(const std::string &a, const std::string &b) const {
     std::string sa = a;
@@ -150,12 +154,38 @@ class HttpBodyRequest : public HttpRequestBase<HttpBodyRequest> {
   std::string body_ = "";
 };
 
+class HttpFileRequest : public HttpRequestBase<HttpFileRequest> {
+public:
+  HttpFileRequest(HttpClient &client, std::string URL, std::string filename,
+                  curl_off_t file_size)
+      : HttpRequestBase(client, std::move(URL), HttpMethod::Put),
+        filename_(std::move(filename)), file_size_(file_size) {}
+
+  HttpFileRequest &progress(UploadProgressCallback callback) {
+    progress_callback_ = std::move(callback);
+    return *this;
+  }
+
+  const std::string &getFilename() const { return filename_; }
+  curl_off_t getFileSize() const { return file_size_; }
+  const UploadProgressCallback &getProgressCallback() const {
+    return progress_callback_;
+  }
+  std::expected<HttpResponse, std::string> execute();
+
+private:
+  std::string filename_;
+  curl_off_t file_size_;
+  UploadProgressCallback progress_callback_;
+};
+
 // HttpClient should only focus on handling the cURL handle
 // and making the request (HttpRequest) and returning HttpResponse
 class HttpClient {
   // `execute()` is invoked from the request only
   friend class HttpRequest;
   friend class HttpBodyRequest;
+  friend class HttpFileRequest;
 
 public:
   HttpClient() {
@@ -207,6 +237,11 @@ class HttpClient {
   [[nodiscard]] HttpBodyRequest put(const std::string &URL) {
     return HttpBodyRequest{*this, URL, HttpMethod::Put};
   };
+  [[nodiscard]] HttpFileRequest putFile(const std::string &URL,
+                                        const std::string &filename,
+                                        curl_off_t file_size) {
+    return HttpFileRequest{*this, URL, filename, file_size};
+  };
   [[nodiscard]] HttpBodyRequest del(const std::string &URL) {
     return HttpBodyRequest{*this, URL, HttpMethod::Delete};
   };
@@ -228,6 +263,8 @@ class HttpClient {
   std::expected<HttpResponse, std::string>
   execute_post(HttpBodyRequest &request);
   std::expected<HttpResponse, std::string>
+  execute_upload(HttpFileRequest &request);
+  std::expected<HttpResponse, std::string>
   execute_delete(HttpBodyRequest &request);
 
   const std::unordered_map<std::string, std::string> &getHeaders() const {