Skip to content

Commit 19b4953

Browse files
committed
Merge upstream/main into fix/issues-187
Made-with: Cursor
2 parents 6dd4c2e + 6e97ed2 commit 19b4953

49 files changed

Lines changed: 1880 additions & 105 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.env.example

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ FRONTEND_URL=http://localhost:5173
55

66
# Docker Postgres defaults
77
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/accesslayer
8+
APP_SECRET=your_32_character_long_secret_string_here
89

910
GOOGLE_CLIENT_ID=
1011
GOOGLE_CLIENT_SECRET=
@@ -23,6 +24,7 @@ PAYSTACK_PUBLIC_KEY=
2324
# API Configuration
2425
API_VERSION=1.0.0
2526
ENABLE_API_VERSION_HEADER=true
27+
ENABLE_SCHEMA_VERSION_HEADER=true
2628
ENABLE_RESPONSE_TIMING=true
2729
ENABLE_REQUEST_LOGGING=true
2830
BACKGROUND_JOB_LOCK_TTL_MS=300000

CONTRIBUTING.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ Thanks for contributing to the backend for Access Layer, a Stellar-native creato
55
## Before you start
66

77
- Read the [README](./README.md) for context.
8+
- Review the [Backend Domain Model and Endpoint Boundaries](./docs/architecture/domain-boundaries.md).
89
- Review the scoped backlog in [docs/open-source/issue-backlog.md](./docs/open-source/issue-backlog.md).
910
- Keep pull requests limited to one backend issue or one documentation improvement.
1011
- Open a discussion before changing core API shape or background processing architecture.

README.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ The server is responsible for:
1515
- notifications, analytics, and moderation workflows
1616
- access checks for gated off-chain content
1717

18+
See [Backend Domain Model and Endpoint Boundaries](./docs/architecture/domain-boundaries.md) for a technical overview and [API Versioning](./docs/api-versioning.md) for details on schema versioning.
19+
1820
## Tech
1921

2022
- Node.js
@@ -167,7 +169,8 @@ readinessProbe:
167169
168170
## Open source workflow
169171
170-
- Read [CONTRIBUTING.md](./CONTRIBUTING.md) before starting work.
171-
- Browse the maintainer issue inventory in [docs/open-source/issue-backlog.md](./docs/open-source/issue-backlog.md).
172+
- Read the [README](./README.md) for context.
173+
- Review the [Backend Domain Model and Endpoint Boundaries](./docs/architecture/domain-boundaries.md).
174+
- Review the scoped backlog in [docs/open-source/issue-backlog.md](./docs/open-source/issue-backlog.md).
172175
- Review [SECURITY.md](./SECURITY.md) before reporting vulnerabilities.
173176
- Use the issue templates in [`.github/ISSUE_TEMPLATE`](./.github/ISSUE_TEMPLATE) for new scoped work.

docs/api-versioning.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# API and Schema Versioning
2+
3+
The Access Layer Server uses versioning headers to inform clients about the current API version and the expected structure of request bodies.
4+
5+
## Response Headers
6+
7+
### `X-API-Version`
8+
9+
Indicates the current overall version of the API. This is typically used for tracking feature sets and major API releases.
10+
11+
### `X-Schema-Version`
12+
13+
Indicates the active version of the request body schema. This version should be checked by consumers to ensure they are sending request bodies in the format expected by the server.
14+
15+
## Versioning Strategy
16+
17+
Both headers follow [Semantic Versioning (SemVer)](https://semver.org/):
18+
19+
- **MAJOR** version: Breaking changes to the API or schema.
20+
- **MINOR** version: Backwards-compatible new features or additions.
21+
- **PATCH** version: Backwards-compatible bug fixes.
22+
23+
## Expected Consumer Behavior
24+
25+
1. **Check Headers**: Consumers should inspect the `X-Schema-Version` header in API responses.
26+
2. **Schema Alignment**: If the `X-Schema-Version` major version changes, consumers must update their request body structures to match the new schema requirements.
27+
3. **Warning Handling**: Consumers may choose to log warnings if they detect a version mismatch that they haven't yet updated to support.
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Backend Domain Model and Endpoint Boundaries
2+
3+
This document outlines the core backend entities, their relationships, and the boundaries between different modules in the Access Layer Server.
4+
5+
## Domain Model
6+
7+
The following diagram illustrates the core entities and their relationships within the system:
8+
9+
```mermaid
10+
erDiagram
11+
User ||--o| CreatorProfile : "owns"
12+
User ||--o| StellarWallet : "links"
13+
User {
14+
string id PK
15+
string email
16+
string passwordHash
17+
string firstName
18+
string lastName
19+
boolean emailVerified
20+
}
21+
CreatorProfile {
22+
string id PK
23+
string userId FK
24+
string handle
25+
string displayName
26+
string bio
27+
json perks
28+
}
29+
StellarWallet {
30+
string id PK
31+
string userId FK
32+
string address
33+
}
34+
IndexerDLQ {
35+
string id PK
36+
string jobType
37+
json payload
38+
string failureReason
39+
}
40+
AuditEvent {
41+
string id PK
42+
string actor
43+
string action
44+
string target
45+
string targetId
46+
json metadata
47+
}
48+
```
49+
50+
### Core Entities
51+
52+
1. **User**: Represents a registered user. Holds authentication and basic profile data.
53+
2. **CreatorProfile**: Represents the creator persona of a user. Tied to a specific handle and contains metadata like bio and perks.
54+
3. **StellarWallet**: Links a user to their Stellar public address. Used for identity verification and ownership checks.
55+
4. **IndexerDLQ**: Stores failed indexing jobs from the Stellar blockchain for manual review or reprocessing.
56+
5. **AuditEvent**: A generic log for significant actions occurring in the system.
57+
58+
## Module Boundaries
59+
60+
The server is organized into feature-based modules under `src/modules/`. Each module is responsible for its own business logic, routes, and (where applicable) data validation.
61+
62+
### Major Route Groups
63+
64+
| Module | Responsibility | Primary Entities |
65+
| :--------- | :---------------------------------------------------------------------------- | :--------------- |
66+
| `auth` | User registration, login, session management, and password resets. | `User` |
67+
| `creators` | Public and private creator profile management, including stats and discovery. | `CreatorProfile` |
68+
| `wallet` | Linking and verifying Stellar wallets. | `StellarWallet` |
69+
| `admin` | Internal management tools and system monitoring. | All |
70+
| `health` | System health checks and status monitoring. | N/A |
71+
72+
### Cross-Module Rules
73+
74+
To ensure a maintainable and decoupled architecture, the following rules apply:
75+
76+
1. **No Direct Database Access**: Modules should not directly query Prisma models belonging to other modules if a service/utility exists.
77+
2. **Shared Utilities**: Common logic (e.g., mail sending, logging, pagination) belongs in `src/utils/` and can be used by any module.
78+
3. **Constants**: Shared configuration and string constants belong in `src/constants/`.
79+
4. **Types**: Cross-cutting TypeScript types belong in `src/types/`.
80+
81+
### Interaction Patterns
82+
83+
- **Initialization**: `src/app.ts` assembles the modules and registers global middlewares.
84+
- **Data Sharing**: If a module needs data from another (e.g., `creators` needing user info), it should use the Prisma client (which is shared) but respect the logical boundaries defined in the schema files.

docs/indexer/EVENT_PROCESSING.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Chain Event Processing
2+
3+
The indexer processes events from the blockchain to update the read models and activity feeds. To ensure data consistency and prevent duplicate processing, the following strategies are employed.
4+
5+
## 1. Deduplication
6+
7+
Before processing a batch of events, they should be deduped based on their unique identifier on the chain: `transactionHash` and `eventIndex`.
8+
9+
The `dedupeChainEvents` helper in `src/utils/indexer-dedupe.utils.ts` provides this functionality.
10+
11+
### Example Usage:
12+
13+
```typescript
14+
import { dedupeChainEvents } from '../utils/indexer-dedupe.utils';
15+
16+
const rawEvents = fetchEventsFromChain();
17+
const uniqueEvents = dedupeChainEvents(rawEvents);
18+
// Proceed with processing uniqueEvents
19+
```
20+
21+
## 2. Idempotency
22+
23+
Event handlers must be idempotent. This means that processing the same event multiple times should have the same effect as processing it once.
24+
25+
### Strategies for Idempotency:
26+
27+
- **Database Upserts**: Use Prisma's `upsert` or `update` with unique constraints where possible.
28+
- **State Check**: Before applying a change (like incrementing a balance), verify if the event has already been accounted for (e.g. by checking a `lastProcessedLedger` or a specific event log).
29+
- **Atomic Transactions**: Ensure that all changes related to an event are committed in a single database transaction.
30+
31+
## 3. Error Handling
32+
33+
If an event fails to process after multiple retries, it is moved to the [Dead-Letter Queue (DLQ)](./DLQ_WORKFLOW.md) for manual investigation.

docs/read-model-rebuild.md

Lines changed: 28 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -46,38 +46,60 @@ Source of truth (events / primary tables)
4646
1. **Pause or fence the live indexer** that writes to the read model, or ensure it is idempotent enough to run concurrently with the rebuild.
4747

4848
2. **Truncate or soft-delete** the stale read-model rows:
49+
4950
```sql
5051
TRUNCATE TABLE creator_ownership_reads;
5152
-- or, for an incremental rebuild without full downtime:
5253
UPDATE creator_ownership_reads SET rebuild_pending = true;
5354
```
5455

5556
3. **Run the rebuild script** (location: `scripts/rebuild-read-model.sh` once implemented):
57+
5658
```bash
5759
pnpm ts-node src/scripts/rebuild-read-model.ts --model=creator_ownership_reads
5860
```
61+
5962
The script must process records in batches (recommended batch size: 500) and log progress at each checkpoint.
6063

6164
4. **Verify output:**
65+
6266
```sql
6367
SELECT COUNT(*) FROM creator_ownership_reads;
6468
-- Compare to expected count derived from source tables
6569
```
70+
6671
Spot-check a sample of records against the source of truth.
6772

6873
5. **Resume the live indexer.** If fenced, lift the fence. If the indexer was paused, restart it and confirm it picks up from the correct cursor position.
6974

7075
## Expected duration
7176

72-
| Table size | Estimated rebuild time |
73-
|---|---|
74-
| < 10k rows | < 1 minute |
75-
| 10k – 100k rows | 2–10 minutes |
76-
| 100k – 1M rows | 15–60 minutes |
77-
| > 1M rows | Plan for multi-hour window; use batched pagination |
77+
| Table size | Estimated rebuild time |
78+
| --------------- | -------------------------------------------------- |
79+
| < 10k rows | < 1 minute |
80+
| 10k – 100k rows | 2–10 minutes |
81+
| 100k – 1M rows | 15–60 minutes |
82+
| > 1M rows | Plan for multi-hour window; use batched pagination |
7883

7984
Duration depends on DB instance size, network, and whether indexes are rebuilt inline or deferred.
8085

86+
## Indexer replay dry-run
87+
88+
Use the admin replay endpoint in dry-run mode to validate replay inputs without producing audit-write side effects.
89+
90+
```bash
91+
curl -X POST "$API_BASE_URL/admin/indexer/replay" \
92+
-H "Content-Type: application/json" \
93+
-H "x-admin-id: <admin-id>" \
94+
-d '{"startLedger": 123456, "dryRun": true}'
95+
```
96+
97+
Notes:
98+
99+
- `dryRun` defaults to `false`; omit it to run a normal replay initiation.
100+
- In dry-run mode, the response includes `dryRun: true` and no audit event is written.
101+
- `startLedger` must be a positive integer in both dry-run and normal mode.
102+
81103
## Rollback guidance
82104

83105
If the rebuild produces incorrect data:

prisma/schema/activity.prisma

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
// prisma/schema/activity.prisma
2+
3+
enum ActivityType {
4+
CREATOR_REGISTERED
5+
KEY_BOUGHT
6+
KEY_SOLD
7+
PROFILE_UPDATED
8+
}
9+
10+
model Activity {
11+
id String @id @default(cuid())
12+
type ActivityType
13+
14+
// Actor who performed the action (wallet address or user ID)
15+
actor String
16+
17+
// Optional creator associated with this activity
18+
creatorId String?
19+
20+
// Optional target of the activity (e.g., target wallet address)
21+
target String?
22+
23+
// Payload for event-specific data (e.g., price, amount, previous values)
24+
payload Json
25+
26+
createdAt DateTime @default(now())
27+
28+
@@index([creatorId])
29+
@@index([actor])
30+
@@index([type])
31+
}

prisma/schema/creator.prisma

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ model CreatorProfile {
99
avatarUrl String?
1010
perkSummary String?
1111
isVerified Boolean @default(false)
12+
perks Json?
1213
createdAt DateTime @default(now())
1314
updatedAt DateTime @updatedAt
1415

prisma/schema/ownership.prisma

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
// prisma/schema/ownership.prisma
2+
3+
model KeyOwnership {
4+
id String @id @default(cuid())
5+
6+
// The wallet address of the owner
7+
ownerAddress String
8+
9+
// The ID or handle of the creator whose keys are owned
10+
creatorId String
11+
12+
// The amount of keys owned
13+
balance Decimal @default(0)
14+
15+
createdAt DateTime @default(now())
16+
updatedAt DateTime @updatedAt
17+
18+
@@unique([ownerAddress, creatorId])
19+
@@index([ownerAddress])
20+
@@index([creatorId])
21+
}

0 commit comments

Comments
 (0)