Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
9888798
initial commit
Umang01-hash Jul 28, 2025
0a8c265
add traces metrics and logs
Umang01-hash Jul 28, 2025
a1799e9
Merge remote-tracking branch 'origin/development' into en/db_resolver
Umang01-hash Jul 29, 2025
c75b89e
Merge remote-tracking branch 'origin/development' into en/db_resolver
Umang01-hash Jul 29, 2025
75956df
resolve linters
Umang01-hash Jul 29, 2025
2d4c32d
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 4, 2025
7baeb33
add mocks and tests
Umang01-hash Aug 4, 2025
547602c
improve test coverage
Umang01-hash Aug 5, 2025
686ada4
build(deps): bump go.opentelemetry.io/otel/exporters/prometheus
dependabot[bot] Aug 5, 2025
847ecf5
build(deps): bump google.golang.org/api from 0.243.0 to 0.244.0
dependabot[bot] Aug 5, 2025
af78d89
build(deps): bump gofr.dev in /examples/using-add-filestore
dependabot[bot] Aug 5, 2025
006ad33
fix connection issues of replica's
Umang01-hash Aug 5, 2025
2a3c0c6
optimize implementation for performance
Umang01-hash Aug 6, 2025
2aa7e57
fix performance bottlenecks
Umang01-hash Aug 7, 2025
552f580
add documentation
Umang01-hash Aug 7, 2025
cd6bed2
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 7, 2025
d3a342e
fix linters
Umang01-hash Aug 7, 2025
c4f1a23
resolve small issue in external_db.go
Umang01-hash Aug 7, 2025
f38c2ef
fix import in datasources.go
Umang01-hash Aug 7, 2025
6f22477
fix import issue in datasources.go
Umang01-hash Aug 7, 2025
a264cd5
retry import fixing
Umang01-hash Aug 7, 2025
c448892
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 8, 2025
6b66f6b
resolve reveiw comments
Umang01-hash Aug 8, 2025
2b9c180
enforce dbresovler to use go 1.24.0
Umang01-hash Aug 12, 2025
11a3a07
enforce dbresovler to use go 1.24.0
Umang01-hash Aug 12, 2025
64236a9
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 13, 2025
ee93394
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 14, 2025
eae293b
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 14, 2025
6419c98
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 14, 2025
da15a7d
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 19, 2025
17a33c4
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 20, 2025
f5d0101
sort go.work dependencies
Umang01-hash Aug 20, 2025
02cc07a
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 25, 2025
50ed2aa
resolve review comments
Umang01-hash Aug 25, 2025
1340259
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 26, 2025
0a08490
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 29, 2025
b379e4e
resolve PR review comments
Umang01-hash Aug 29, 2025
33d8e50
add new configs for db_replica_users and passwords
Umang01-hash Aug 29, 2025
b53ab3b
Merge branch 'development' into en/db_resolver
coolwednesday Aug 29, 2025
7ef7fc5
add new configs in docs
Umang01-hash Aug 29, 2025
a8440e2
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 29, 2025
2adf3f0
Merge remote-tracking branch 'origin/en/db_resolver' into en/db_resolver
Umang01-hash Aug 29, 2025
ae3b0d5
resolve linters
Umang01-hash Aug 29, 2025
64cf385
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 30, 2025
de83572
Merge branch 'development' into en/db_resolver
Umang01-hash Sep 1, 2025
b255ed2
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Sep 23, 2025
e66ec29
resolve review comments
Umang01-hash Sep 23, 2025
6700709
unexport strategy constants
Umang01-hash Sep 23, 2025
632a03b
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Sep 25, 2025
6b90162
go mod tidy
Umang01-hash Sep 25, 2025
f029db2
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Oct 28, 2025
adff5ae
Merge branch 'development' into en/db_resolver
Umang01-hash Nov 4, 2025
c28ef91
basic implementation for route based query resolving
Umang01-hash Oct 31, 2025
98d0693
update implementation and add AddSQL and GetSQL method to replace con…
Umang01-hash Nov 11, 2025
fa3e9d4
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Nov 11, 2025
7e71a81
add tests and resolve linters
Umang01-hash Nov 11, 2025
99d0c66
revert unwanted changes
Umang01-hash Nov 12, 2025
b18fbe8
Merge branch 'development' into en/db_resolver
Umang01-hash Nov 12, 2025
a975492
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Nov 13, 2025
a251d70
resolve review comments
Umang01-hash Nov 13, 2025
b65a2f7
Merge remote-tracking branch 'refs/remotes/origin/en/db_resolver' int…
Umang01-hash Nov 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 131 additions & 0 deletions docs/quick-start/connecting-mysql/page.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,3 +239,134 @@ Now when we access {% new-tab-link title="http://localhost:9000/customer" href="
```

**Note:** When using PostgreSQL or Supabase, you may need to use `$1` instead of `?` in SQL queries, depending on your driver configuration.

### Enabling Read/Write Splitting in MySQL (DBResolver)
GoFr provides built-in support for read/write splitting using its `DBRESOLVER` module for **MySQL**.
This feature automatically routes requests to the **primary database** or **read replicas** based on:

- **HTTP Method**: Write operations (`POST`, `PUT`, `PATCH`, `DELETE`) → Primary | Read operations (`GET`, `HEAD`, `OPTIONS`) → Replicas
- **Route Configuration**: Force specific routes to always use the primary database for strong consistency

#### Installation

Import the GoFr's dbresolver for MySQL:

```shell
go get gofr.dev/pkg/gofr/datasource/dbresolver@latest
```

#### Configuration

**1. Environment Variables**

Add replica connection details to your .env file:

```editorconfig
# Primary database
DB_HOST=localhost
DB_PORT=3306
DB_USER=root
DB_PASSWORD=root123
DB_NAME=test_db
DB_DIALECT=mysql

# Replica configuration
DB_REPLICA_HOSTS=localhost,replica1,replica2
DB_REPLICA_PORTS=3307,3308,3309
DB_REPLICA_USERS=readonly1,readonly2,readonly3
DB_REPLICA_PASSWORDS=pass1,pass2,pass3
```

**2. Initialize DBResolver**

After importing the package, you can configure the DBResolver in your GoFr application using the `AddDBResolver` method.
You can choose the load balancing strategy and enable fallback to primary:

```go
package main

import (
"gofr.dev/pkg/gofr"
"gofr.dev/pkg/gofr/datasource/dbresolver"
)

type Customer struct {
ID int `db:"id"`
Name string `db:"name"`
}

func main() {
a := gofr.New()

// Initialize DB resolver with default settings
err := dbresolver.InitDBResolver(a, dbresolver.Config{
Strategy: dbresolver.StrategyRoundRobin, // use round-robin strategy or random strategy
ReadFallback: true, // allow reads on primary if all replicas are down
MaxFailures: 3, // number of allowed failures before marking a replica as down
TimeoutSec: 30, // timeout for marking a replica as down
PrimaryRoutes: []string{"/admin", "/api/payments/*"}, // routes that should go to primary
})
if err != nil {
a.Logger().Errorf("failed to initialize db resolver: %v", err)
}

// Read endpoint - goes to replica
a.GET("/customers", func(c *gofr.Context) (interface{}, error) {
var customers []Customer

c.SQL.Select(c, &customers, "SELECT id, name FROM customers")

return customers, err
})

// Write endpoint - goes to primary
a.POST("/customers", func(c *gofr.Context) (interface{}, error) {
var customer Customer

c.Bind(&customer)

_, err := c.SQL.Exec("INSERT INTO customers (name) VALUES (?)", customer.Name)

return customer, err
})

// Admin endpoint - forced to primary
a.GET("/admin/customers", func(c *gofr.Context) (interface{}, error) {
var customers []Customer

c.SQL.Select(c, &customers, "SELECT id, name FROM customers")

return customers, err
})

a.Run()
}
```

**3. Connection Pool Tuning (Optional)**

By default, replica pools are auto-scaled based on primary settings:

```
# Defaults (automatically calculated)
DB_MAX_IDLE_CONNECTION=2 → Replicas: 8 (2 × 4)
DB_MAX_OPEN_CONNECTION=20 → Replicas: 40 (20 × 2)
```

Override with:

```
DB_REPLICA_MAX_IDLE_CAP=100
DB_REPLICA_MIN_IDLE=5
DB_REPLICA_DEFAULT_IDLE=15

DB_REPLICA_MAX_OPEN_CAP=500
DB_REPLICA_MIN_OPEN=20
DB_REPLICA_DEFAULT_OPEN=150

```

**Benefits**
- Performance: Offloads read traffic from the primary, reducing latency.
- Scalability: Easily scale reads by adding more replicas.
- Resilience: Ensures high availability through automatic fallback.
61 changes: 61 additions & 0 deletions docs/references/configs/page.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,67 @@ This document lists all the configuration options supported by the GoFr framewor

---

- DB_REPLICA_HOSTS
- Comma-separated list of replica database hosts. Used for read replicas.
- None

---

- DB_REPLICA_PORTS
- Comma-separated list of replica database ports. Used for read replicas.
- None

---

- DB_REPLICA_USERS
- Comma-separated list of replica database users. Used for read replicas.
- None

---

- DB_REPLICA_PASSWORDS_
- Comma-separated list of replica database passwords. Used for read replicas.
- None

---

- DB_REPLICA_MAX_IDLE_CONNECTIONS
- Maximum idle connections allowed for a replica
- 50

---

- DB_REPLICA_MIN_IDLE_CONNECTIONS
- Minimum idle connections for a replica
- 10

---

- DB_REPLICA_DEFAULT_IDLE_CONNECTIONS
- Idle connections used if no primary setting is provided
- 10

---

- DB_REPLICA_MAX_OPEN_CONNECTIONS
- Maximum open connections allowed for a replica
- 200

---

- DB_REPLICA_MIN_OPEN_CONNECTIONS
- Minimum open connections for a replica
- 50

---

- DB_REPLICA_DEFAULT_OPEN_CONNECTIONS
- Open connections used if no primary setting is provided
- 100

---


- DB_CHARSET
- The character set for database connection
- utf8
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ require (
)

require (
cloud.google.com/go v0.120.0 // indirect
cloud.google.com/go v0.121.1 // indirect
cloud.google.com/go/auth v0.17.0 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.120.0 h1:wc6bgG9DHyKqF5/vQvX1CiZrtHnxJjBlKUyF9nP6meA=
cloud.google.com/go v0.120.0/go.mod h1:/beW32s8/pGRuj4IILWQNd4uuebeT4dkOhKmkfit64Q=
cloud.google.com/go v0.121.1 h1:S3kTQSydxmu1JfLRLpKtxRPA7rSrYPRPEUmL/PavVUw=
cloud.google.com/go v0.121.1/go.mod h1:nRFlrHq39MNVWu+zESP2PosMWA0ryJw8KUBZ2iZpxbw=
cloud.google.com/go/auth v0.17.0 h1:74yCm7hCj2rUyyAocqnFzsAYXgJhrG26XCFimrc/Kz4=
cloud.google.com/go/auth v0.17.0/go.mod h1:6wv/t5/6rOPAX4fJiRjKkJCvswLwdet7G8+UGXt7nCQ=
cloud.google.com/go/auth/oauth2adapt v0.2.8 h1:keo8NaayQZ6wimpNSmW5OPc283g65QNIiLpZnkHRbnc=
Expand Down
1 change: 1 addition & 0 deletions go.work
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ use (
./pkg/gofr/datasource/cassandra
./pkg/gofr/datasource/clickhouse
./pkg/gofr/datasource/couchbase
./pkg/gofr/datasource/dbresolver
./pkg/gofr/datasource/dgraph
./pkg/gofr/datasource/elasticsearch
./pkg/gofr/datasource/file/ftp
Expand Down
7 changes: 7 additions & 0 deletions pkg/gofr/container/datasources.go
Original file line number Diff line number Diff line change
Expand Up @@ -786,6 +786,13 @@ type CouchbaseProvider interface {
provider
}

// DBResolverProvider defines an interface for SQL read/write splitting providers.
type DBResolverProvider interface {
GetResolver() DB

provider
}

// InfluxDB defines the operations required to interact with an InfluxDB instance.
type InfluxDB interface {
// CreateOrganization create new bucket in the influxdb
Expand Down
125 changes: 86 additions & 39 deletions pkg/gofr/container/mock_datasources.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading