Skip to content

feat: WebBaser #1363

Open
KeaxD wants to merge 27 commits intoWebOfTrust:mainfrom
KeaxD:feat-WebBaser
Open

feat: WebBaser #1363
KeaxD wants to merge 27 commits intoWebOfTrust:mainfrom
KeaxD:feat-WebBaser

Conversation

@KeaxD
Copy link
Copy Markdown
Contributor

@KeaxD KeaxD commented Mar 26, 2026

This PR attemps to implement WebBaser. Currently, WebBaser passes most tests from test_basing.py. Subdbs are being instantiated and are correctly using IoSet, OnIoSet methods from Webdbing.py. However, there are still work to be done:

  • Implementing a WebBaser Doer
  • .clean() function still needs to be implemented/tested, I currently have a stub for it with my attempt to write it. However, I still need to read more about IndexedDB.
  • More tests coverage for other functions of WebBaser.

There are also some things to note:

Subkey Naming Anomalies

Two sub‑DBs use leading‑dot subkeys:
self.epath = IoSetSuber(db=self, subkey=".epath") instead of epath.
self.essrs = CesrIoSetSuber(db=self, subkey=".essrs", klas=Texter) instead of essrs.
I am not sure if this was by design or a typo.

IoSetSuber methods and IoDupSuber methods differences

IoDupSuber Pin:

# edge case: pin with mixed types
assert db.pses.pin(keys=key, vals=[b'A', 'A', memoryview(b'A')]) == True
assert db.pses.get(keys=key) == ['A', 'A', 'A'] # All 3 values are added

# pin with a different list
assert db.pses.pin(keys=key, vals=[b'x', b'y']) == True
assert db.pses.get(keys=key) == ['x', 'y']  # previous values removed

# pin with empty list (valid use case)
assert db.pses.pin(keys=key, vals=[]) == False  # nothing to pin
assert db.pses.get(keys=key) == []  # key cleared

IoSetSuber Pin:

# edge case: pin with mixed types
assert baser.pses.pin(keys=key, vals=[b'A', 'A', memoryview(b'A')]) == True
assert baser.pses.get(keys=key) == ['A'] # only one value gets added

# pin with a different list
assert baser.pses.pin(keys=key, vals=[b'x', b'y']) == True
assert baser.pses.get(keys=key) == ['x', 'y']  # previous values removed

# pin with empty list (valid use case)
assert baser.pses.pin(keys=key, vals=[]) == False  # nothing to pin
assert baser.pses.get(keys=key) == ['x', 'y']  # previous values are still here

IoDupSuber Put:

# edge case: add different types of vals (bytes and string)
assert db.pses.put(keys=key, vals=[b'a','a']) == True
assert db.pses.get(keys=key) == ['a', 'a'] # both value added because _ser produces different bytes

IoSetSuber Pin:

# edge case: add different types of vals (bytes and string)
assert baser.pses.put(keys=key, vals=[b'a','a']) == True
assert baser.pses.get(keys=key) == ['a'] # only 1 value added

getLastIter for IoDupSuber and IoSetSuber are different:

OnIoSetSuber: getIoSetLastItemIterAll which Iterates over every last added ioset entry at every effective key starting at key greater or equal to key.

OnIoDupSuber: getOnIoDupLastValIter which returns iterator of val of last insertion ordered duplicate at each key over all ordinal numbered onkeys in db with same  key

So in the following test, Kels being an IoSetSuber, getLastIter Iterates over every last added ioset entry at every effective key starting at key greater or equal to key. Since the previous key (BWzwEHH) is lexicographically superior to (B4ejhccWzwEHH), when we start to iterate at B4ejhccWzwEHH, we also return the next key.

backend = FakeStorageBackend()
       baser = WebBaser()

       await baser.reopen(storageOpener=backend.open)

       assert baser.opened
       assert baser.name == "main"


       preb = 'BWzwEHHzq7K0gzQPYGGwTmuupUhPx5_yZ-Wk1x4ejhcc'.encode("utf-8")
       digb = 'EGAPkzNZMtX-QiVgbRbyAIZGoXvbGv9IPb0foWTZvI_4'.encode("utf-8")
       sn = 3
       vs = versify(kind=Kinds.json, size=20)
       assert vs == 'KERI10JSON000014_'

       ked = dict(vs=vs, pre=preb.decode("utf-8"),
               sn="{:x}".format(sn),
               ilk="rot",
               dig=digb.decode("utf-8"))
       skedb = json.dumps(ked, separators=(",", ":"), ensure_ascii=False).encode("utf-8")
       assert skedb == (b'{"vs":"KERI10JSON000014_","pre":"BWzwEHHzq7K0gzQPYGGwTmuupUhPx5_yZ-Wk1x4ejhc'
                       b'c","sn":"3","ilk":"rot","dig":"EGAPkzNZMtX-QiVgbRbyAIZGoXvbGv9IPb0foWTZvI_4"'
                       b'}')

       # test kels getAllIter
       sn = 0
       vals0 = [skedb]
       assert baser.kels.add(keys=preb, on=sn, val=vals0[0]) == True

       vals1 = [b"mary", b"peter", b"john", b"paul"]
       sn += 1
       for val in vals1:
           assert baser.kels.add(keys=preb, on=sn, val=val) == True

       vals2 = [b"dog", b"cat", b"bird"]
       sn += 1
       for val in vals2:
           assert baser.kels.add(keys=preb, on=sn, val=val) == True

       vals = list(baser.kels.getAllIter(keys=preb))
       allvals = [v.decode("utf-8") for v in (vals0 + vals1 + vals2)]
       assert vals == allvals

       # test kels getLastIter
       preb = 'B4ejhccWzwEHHzq7K0gzQPYGGwTmuupUhPx5_yZ-Wk1x'.encode("utf-8")
       sn = 0
       
       vals0 = [skedb]
       assert baser.kels.add(keys=preb, on=sn, val=vals0[0]) == True

       vals1 = [b"mary", b"peter", b"john", b"paul"]
       sn += 1
       for val in vals1:
           assert baser.kels.add(keys=preb, on=sn, val=val) == True

       vals2 = [b"dog", b"cat", b"bird"]
       sn += 1
       for val in vals2:
           assert baser.kels.add(keys=preb, on=sn, val=val) == True
       vals = list(baser.kels.getLastIter(keys=preb))
       # Kels being an IoSetSuber, getLastIter calls getIoSetLastItemIterAll 
       # which Iterates over every last added ioset entry at every effective key
       # starting at key greater or equal to key so the values from the previous tests are 
       # yielded here too.
       lastvals = ['{"vs":"KERI10JSON000014_","pre":"BWzwEHHzq7K0gzQPYGGwTmuupUhPx5_yZ-Wk1x4ejhcc","sn":"3","ilk":"rot","dig":"EGAPkzNZMtX-QiVgbRbyAIZGoXvbGv9IPb0foWTZvI_4"}', 'paul', 'bird', 
                   '{"vs":"KERI10JSON000014_","pre":"BWzwEHHzq7K0gzQPYGGwTmuupUhPx5_yZ-Wk1x4ejhcc","sn":"3","ilk":"rot","dig":"EGAPkzNZMtX-QiVgbRbyAIZGoXvbGv9IPb0foWTZvI_4"}', 'paul', 'bird']

       assert vals == lastvals

@KeaxD KeaxD marked this pull request as draft March 26, 2026 18:33
@SmithSamuelM
Copy link
Copy Markdown
Collaborator

The naming subdivision with leading dots is a typo. The convention is to use trailing dots to cleanly separate the name space of the subdb from keys inside the subdb. This is beacause LMDB allows whole database access (ignoring subdbs) and using the trailing dots as a convention preserves the ability to distinquish subdb boundaries.

@KeaxD KeaxD marked this pull request as ready for review March 27, 2026 16:36
@KeaxD KeaxD marked this pull request as draft March 27, 2026 18:41
@KeaxD KeaxD marked this pull request as ready for review March 27, 2026 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants