Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions evals/cosmosdb-best-practices/tasks/model-ttl-expiration.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
id: model-ttl-expiration
name: Data Modeling - Use TTL for automatic data expiration
description: |
Test that the skill recommends Cosmos DB TTL instead of scheduled cleanup
jobs for data with a natural retention window.
tags:
- modeling
- ttl
- retention
inputs:
prompt: "I store short-lived session tokens and temporary cache entries in Cosmos DB. Should I run a nightly job to delete expired records, or is there a built-in way to expire them automatically?"
expected:
outcomes:
- type: task_completed
87 changes: 85 additions & 2 deletions skills/cosmosdb-best-practices/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ Performance optimization and best practices guide for Azure Cosmos DB applicatio
- 1.8 [Reference Data When Items Grow Large](#18-reference-data-when-items-grow-large)
- 1.9 [Use ID references with transient hydration for document relationships](#19-use-id-references-with-transient-hydration-for-document-relationships)
- 1.10 [Version Your Document Schemas](#110-version-your-document-schemas)
- 1.11 [Use Type Discriminators for Polymorphic Data](#111-use-type-discriminators-for-polymorphic-data)
- 1.11 [Use TTL for Automatic Data Expiration](#111-use-ttl-for-automatic-data-expiration)
- 1.12 [Use Type Discriminators for Polymorphic Data](#112-use-type-discriminators-for-polymorphic-data)
2. [Partition Key Design](#2-partition-key-design) — **CRITICAL**
- 2.1 [Plan for 20GB Logical Partition Limit](#21-plan-for-20gb-logical-partition-limit)
- 2.2 [Distribute Writes to Avoid Hot Partitions](#22-distribute-writes-to-avoid-hot-partitions)
Expand Down Expand Up @@ -1317,7 +1318,89 @@ Always increment version when:

Reference: [Schema evolution in Cosmos DB](https://learn.microsoft.com/azure/cosmos-db/nosql/modeling-data)

### 1.11 Use Type Discriminators for Polymorphic Data
### 1.11 Use TTL for Automatic Data Expiration

**Impact: MEDIUM** (removes stale data without custom cleanup jobs)

## Use TTL for Automatic Data Expiration

Use Azure Cosmos DB time to live (TTL) for data that has a natural retention window, such as session tokens, event logs, temporary cache entries, invitations, one-time codes, or short-lived processing state. TTL lets the service expire items automatically instead of requiring a scheduled cleanup job that scans and deletes old records.

TTL is configured in seconds. The expiration countdown is based on the item's last modified timestamp (`_ts`), so updating an item resets its TTL window.

**Incorrect (scheduled cleanup job scans and deletes expired items):**

```csharp
// Anti-pattern: periodic cleanup query scans old items and deletes them one by one.
var query = new QueryDefinition(
"SELECT * FROM c WHERE c.type = 'session' AND c.expiresAt < @now")
.WithParameter("@now", DateTimeOffset.UtcNow);

using var iterator = container.GetItemQueryIterator<Session>(query);
while (iterator.HasMoreResults)
{
foreach (var session in await iterator.ReadNextAsync())
{
await container.DeleteItemAsync<Session>(
session.Id,
new PartitionKey(session.UserId));
}
}
```

```json
{
"id": "session-123",
"userId": "user-42",
"type": "session",
"expiresAt": "2026-06-11T13:00:00Z"
}
```

**Correct (enable TTL and set per-item expiration):**

```csharp
// Enable TTL on the container without a default expiration.
// Items expire only when they include their own ttl value.
await database.DefineContainer("sessions", "/userId")
.WithDefaultTimeToLive(-1)
.CreateAsync();

var session = new
{
id = "session-123",
userId = "user-42",
type = "session",
ttl = 3600, // Expire one hour after the item is created or last modified.
createdAt = DateTimeOffset.UtcNow
};

await container.CreateItemAsync(session, new PartitionKey(session.userId));
```

```json
{
"id": "session-123",
"userId": "user-42",
"type": "session",
"ttl": 3600,
"createdAt": "2026-06-11T12:00:00Z"
}
```

Use the right TTL mode for the retention pattern:

- Leave container TTL unset or `null` when items should never expire automatically.
- Set container `DefaultTimeToLive` to a positive number when every item should expire after the same retention period.
- Set container `DefaultTimeToLive` to `-1` when TTL should be enabled but only specific items should expire.
- Set item-level `ttl` to a positive number to override the container default for that item.
- Set item-level `ttl` to `-1` for items that should not expire in a TTL-enabled container.

TTL is best for automatic retention, not exact scheduling. Expired items stop appearing in query results after the TTL expires, but physical deletion happens asynchronously in the background. In provisioned throughput accounts, TTL deletes use leftover RUs; in serverless accounts, deletes are charged like delete item operations.

Reference: [Time to Live in Azure Cosmos DB](https://learn.microsoft.com/azure/cosmos-db/time-to-live)

### 1.12 Use Type Discriminators for Polymorphic Data

**Impact: MEDIUM** (enables efficient single-container design)

Expand Down
84 changes: 84 additions & 0 deletions skills/cosmosdb-best-practices/rules/model-ttl-expiration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
title: Use TTL for Automatic Data Expiration
impact: MEDIUM
impactDescription: removes stale data without custom cleanup jobs
tags: model, ttl, expiration, retention, cleanup
---

## Use TTL for Automatic Data Expiration

Use Azure Cosmos DB time to live (TTL) for data that has a natural retention window, such as session tokens, event logs, temporary cache entries, invitations, one-time codes, or short-lived processing state. TTL lets the service expire items automatically instead of requiring a scheduled cleanup job that scans and deletes old records.

TTL is configured in seconds. The expiration countdown is based on the item's last modified timestamp (`_ts`), so updating an item resets its TTL window.

**Incorrect (scheduled cleanup job scans and deletes expired items):**

```csharp
// Anti-pattern: periodic cleanup query scans old items and deletes them one by one.
var query = new QueryDefinition(
"SELECT * FROM c WHERE c.type = 'session' AND c.expiresAt < @now")
.WithParameter("@now", DateTimeOffset.UtcNow);

using var iterator = container.GetItemQueryIterator<Session>(query);
while (iterator.HasMoreResults)
{
foreach (var session in await iterator.ReadNextAsync())
{
await container.DeleteItemAsync<Session>(
session.Id,
new PartitionKey(session.UserId));
}
}
```

```json
{
"id": "session-123",
"userId": "user-42",
"type": "session",
"expiresAt": "2026-06-11T13:00:00Z"
}
```

**Correct (enable TTL and set per-item expiration):**

```csharp
// Enable TTL on the container without a default expiration.
// Items expire only when they include their own ttl value.
await database.DefineContainer("sessions", "/userId")
.WithDefaultTimeToLive(-1)
.CreateAsync();

var session = new
{
id = "session-123",
userId = "user-42",
type = "session",
ttl = 3600, // Expire one hour after the item is created or last modified.
createdAt = DateTimeOffset.UtcNow
};

await container.CreateItemAsync(session, new PartitionKey(session.userId));
```

```json
{
"id": "session-123",
"userId": "user-42",
"type": "session",
"ttl": 3600,
"createdAt": "2026-06-11T12:00:00Z"
}
```

Use the right TTL mode for the retention pattern:

- Leave container TTL unset or `null` when items should never expire automatically.
- Set container `DefaultTimeToLive` to a positive number when every item should expire after the same retention period.
- Set container `DefaultTimeToLive` to `-1` when TTL should be enabled but only specific items should expire.
- Set item-level `ttl` to a positive number to override the container default for that item.
- Set item-level `ttl` to `-1` for items that should not expire in a TTL-enabled container.

TTL is best for automatic retention, not exact scheduling. Expired items stop appearing in query results after the TTL expires, but physical deletion happens asynchronously in the background. In provisioned throughput accounts, TTL deletes use leftover RUs; in serverless accounts, deletes are charged like delete item operations.

Reference: [Time to Live in Azure Cosmos DB](https://learn.microsoft.com/azure/cosmos-db/time-to-live)