Skip to content

Commit 62db35a

Browse files
aclark4lifeaclark4life
committed
Code review fixes (3/x)
In the QE topics guide I'm going to explain how to develop a patient model similar to the Python QE tutorial and the encryption test suite. If embedding were possible now, I could present the same example. Since embedded models are a WIP I may either: - design the guide to work around that issue - design the guide to anticipate resolution of that issue In ether case, I'll try to wrap this 24-48 hour docs jam having covered everything I know about QE and Django and having presented it to Django folks in a way we can support.
1 parent c103a1e commit 62db35a

File tree

7 files changed

+87
-74
lines changed

7 files changed

+87
-74
lines changed

‎docs/source/howto/queryable-encryption.rst

Lines changed: 73 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,10 @@ Configuring Queryable Encryption
44

55
.. versionadded:: 5.2.0rc1
66

7+
This guide is similar to the
8+
:doc:`manual:core/queryable-encryption/quick-start` but with some additional
9+
steps required to configure Queryable Encryption with Django MongoDB Backend.
10+
711
.. admonition:: MongoDB requirements
812

913
Queryable Encryption can be used with MongoDB replica sets or sharded
@@ -94,27 +98,72 @@ configure a custom router for Queryable Encryption:
9498
9599
DATABASE_ROUTERS = [EncryptedRouter]
96100
97-
Configuring KMS Providers
98-
=========================
99-
100-
To use Queryable Encryption, you must configure a Key Management Service (KMS)
101-
provider. The KMS provider is responsible for managing the encryption keys used
102-
to encrypt and decrypt data. The following table summarizes the available KMS
103-
provider options and how to configure them:
104-
105-
+-------------------------------------------------------------------------+---------------------------------------+
106-
| :setting:`KMS_CREDENTIALS <DATABASE-KMS-CREDENTIALS>` | A dictionary of Key Management |
107-
| | Service (KMS) credentials |
108-
| | configured in the |
109-
| | :setting:`django:DATABASES` |
110-
| | setting. |
111-
+-------------------------------------------------------------------------+---------------------------------------+
112-
| :class:`kms_providers <pymongo.encryption_options.AutoEncryptionOpts>` | Map of KMS provider credentials and |
113-
| | options. The ``kms_providers`` map |
114-
| | values differ by provider and are |
115-
| | required to access KMS services. |
116-
+-------------------------------------------------------------------------+---------------------------------------+
117-
| ``kms_provider`` | A single KMS provider name |
118-
| | configured in your custom database |
119-
| | router. |
120-
+-------------------------------------------------------------------------+---------------------------------------+
101+
Configuring the Key Management Service (KMS)
102+
============================================
103+
104+
To use Queryable Encryption, you must configure a Key Management Service (KMS).
105+
The KMS is responsible for managing the encryption keys used to encrypt and
106+
decrypt data. The following table summarizes the available KMS configuration
107+
options followed by an example of how to use them.
108+
109+
+-------------------------------------------------------------------------+--------------------------------------------------------+
110+
| :setting:`KMS_CREDENTIALS <DATABASE-KMS-CREDENTIALS>` | A dictionary of Key Management Service (KMS) |
111+
| | credentials configured in the |
112+
| | :setting:`django:DATABASES` setting. |
113+
+-------------------------------------------------------------------------+--------------------------------------------------------+
114+
| :class:`kms_providers <pymongo.encryption_options.AutoEncryptionOpts>` | A dictionary of KMS provider credentials used to |
115+
| | access the KMS with |
116+
| | :setting:`KMS_CREDENTIALS <DATABASE-KMS-CREDENTIALS>`. |
117+
+-------------------------------------------------------------------------+--------------------------------------------------------+
118+
| ``kms_provider`` | A single KMS provider name |
119+
| | configured in your custom database |
120+
| | router. |
121+
+-------------------------------------------------------------------------+--------------------------------------------------------+
122+
123+
Example of KMS configuration with AWS KMS:
124+
125+
.. code-block:: python
126+
127+
from django_mongodb_backend import parse_uri
128+
from pymongo.encryption_options import AutoEncryptionOpts
129+
130+
DATABASES = {
131+
"encrypted": parse_uri(
132+
DATABASE_URL,
133+
options={
134+
"auto_encryption_opts": AutoEncryptionOpts(
135+
key_vault_namespace="keyvault.keyvault",
136+
kms_providers={
137+
"aws": {
138+
"accessKeyId": "your-access-key-id",
139+
"secretAccessKey": "your-secret-access-key",
140+
}
141+
},
142+
)
143+
},
144+
db_name="encrypted",
145+
),
146+
}
147+
148+
DATABASES["encrypted"]["KMS_CREDENTIALS"] = {
149+
"aws": {
150+
"key": os.getenv("AWS_KEY_ARN", ""),
151+
"region": os.getenv("AWS_KEY_REGION", ""),
152+
},
153+
}
154+
155+
156+
class EncryptedRouter:
157+
# ...
158+
def kms_provider(self, model, **hints):
159+
return "aws"
160+
161+
162+
Configuring the ``encrypted_fields_map``
163+
========================================
164+
165+
Configuring the Crypt Shared Library
166+
====================================
167+
168+
You are now ready to :doc:`develop with Queryable Encryption
169+
</topics/queryable-encryption>` in Django MongoDB Backend!

‎docs/source/ref/models/encrypted-fields.rst

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,8 +36,6 @@ Queryable Encryption.
3636
+----------------------------------------+------------------------------------------------------+
3737
| ``EncryptedIntegerField`` | :class:`~django.db.models.IntegerField` |
3838
+----------------------------------------+------------------------------------------------------+
39-
| ``EncryptedPositiveBigIntegerField`` | :class:`~django.db.models.PositiveBigIntegerField` |
40-
+----------------------------------------+------------------------------------------------------+
4139
| ``EncryptedPositiveIntegerField`` | :class:`~django.db.models.PositiveIntegerField` |
4240
+----------------------------------------+------------------------------------------------------+
4341
| ``EncryptedPositiveSmallIntegerField`` | :class:`~django.db.models.PositiveSmallIntegerField` |

‎docs/source/topics/queryable-encryption.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,5 @@ Queryable Encryption
33
====================
44

55
.. versionadded:: 5.2.0rc1
6+
7+
Use :doc:`/ref/models/encrypted-fields` to structure your sensitive data.

‎tests/encryption_/models.py

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@
1111
EncryptedFloatField,
1212
EncryptedGenericIPAddressField,
1313
EncryptedIntegerField,
14-
EncryptedPositiveIntegerField,
1514
EncryptedPositiveSmallIntegerField,
1615
EncryptedSmallIntegerField,
1716
EncryptedTextField,
@@ -77,9 +76,8 @@ class Meta:
7776

7877

7978
class EncryptedNumbers(models.Model):
80-
class Meta:
81-
required_db_features = {"supports_queryable_encryption"}
82-
83-
pos_bigint = EncryptedPositiveIntegerField(queries=EQUALITY_QUERY)
8479
pos_smallint = EncryptedPositiveSmallIntegerField(queries=EQUALITY_QUERY)
8580
smallint = EncryptedSmallIntegerField(queries=EQUALITY_QUERY)
81+
82+
class Meta:
83+
required_db_features = {"supports_queryable_encryption"}

‎tests/encryption_/test_base.py

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
from .models import (
66
Appointment,
77
Billing,
8+
EncryptedNumbers,
89
Patient,
910
PatientPortalUser,
1011
PatientRecord,
@@ -45,3 +46,7 @@ def setUp(self):
4546
is_active=True,
4647
email="[email protected]",
4748
)
49+
EncryptedNumbers.objects.create(
50+
pos_smallint=12345,
51+
smallint=-12345,
52+
)

‎tests/encryption_/test_fields.py

Lines changed: 4 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -81,20 +81,10 @@ def test_patient(self):
8181
records = connections["encrypted"].database.encryption__patientrecord.find()
8282
self.assertTrue("__safeContent__" in records[0])
8383

84-
def test_numeric_fields(self):
85-
"""
86-
Fields that have not been tested elsewhere.
87-
"""
88-
EncryptedNumbers.objects.create(
89-
pos_bigint=1000000,
90-
pos_smallint=12345,
91-
smallint=-12345,
92-
)
93-
94-
obj = EncryptedNumbers.objects.get(pos_bigint=1000000)
84+
def test_pos_small_int(self):
9585
obj = EncryptedNumbers.objects.get(pos_smallint=12345)
96-
obj = EncryptedNumbers.objects.get(smallint=-12345)
97-
98-
self.assertEqual(obj.pos_bigint, 1000000)
9986
self.assertEqual(obj.pos_smallint, 12345)
87+
88+
def test_small_int(self):
89+
obj = EncryptedNumbers.objects.get(smallint=-12345)
10090
self.assertEqual(obj.smallint, -12345)

‎tests/encryption_/test_management.py

Lines changed: 0 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,8 @@
1-
import os
21
from io import StringIO
32

4-
import pymongo
53
from bson import json_util
64
from django.core.management import call_command
7-
from django.db import connections
85
from django.test import modify_settings, override_settings
9-
from pymongo.encryption import AutoEncryptionOpts
106

117
from .routers import TestEncryptedRouter
128
from .test_base import QueryableEncryptionTestCase
@@ -81,32 +77,7 @@ def test_create_new_keys(self):
8177
stdout=out,
8278
)
8379
command_output = json_util.loads(out.getvalue())
84-
85-
# TODO: Until the todo below is fixed, you can test this feature
86-
# manually by uncommenting the following line and pasting the output
87-
# into your client-side `encrypted_fields_map` configuration. You also
88-
# need to temporarily configure server-side encryption for this to
89-
# procedure to work.
90-
# print(command_output)
9180
self._compare_output(
9281
self.expected_patient_record,
9382
command_output["encryption__patientrecord"],
9483
)
95-
96-
# Create a new connection to verify that the keys can be used in a
97-
# client-side configuration to migrate the encrypted fields.
98-
conn_params = connections["encrypted"].get_connection_params()
99-
auto_encryption_opts = AutoEncryptionOpts(
100-
key_vault_namespace="encryption.__keyvault",
101-
kms_providers={"local": {"key": os.urandom(96)}},
102-
encrypted_fields_map=command_output,
103-
)
104-
if conn_params.pop("auto_encryption_opts", False):
105-
# Call MongoClient instead of get_new_connection because
106-
# get_new_connection will return the encrypted connection from the
107-
# connection pool.
108-
with pymongo.MongoClient(**conn_params, auto_encryption_opts=auto_encryption_opts):
109-
call_command("migrate", "--database", "encrypted", verbosity=0)
110-
111-
# TODO: Check the key vault to ensure that the keys created by
112-
# `showencryptedfieldsmap --create-new-keys` are in the key vault.

0 commit comments

Comments
 (0)