Key error fix

[yarko]

This addresses Issue #162; the update from sqlcipher-wheel, in Issue #144 also appears to fix the issue with sigexport working with Python3.13 and later. That change in requirements.lock also appears here.

With a recent update of Signal.app (version 7.44 on iOS, 7.41 on Linux desktop), a Null type record appears, which seems to hold message send/receive/edit histories. This causes a KeyError exception at line 78 of data.py (line 80 in this pull request - I tagged it for easy navigation for myself; feel free to change as you see fit).

Uncommenting data.py: 72allows interactive inspection just before the KeyError exception occurs, using Ipython.
You can %load the following example script to identify the Null records, which are missing a conversationId key:

# coding: utf-8
# %load debug.py
from pprint import pprint

ress = []; cids = []  # plural of 'res' and 'cid', for collection, inspection
notconvo=[]  # entries w/o 'conversationId' keys

for result in c:
    ress.append(json.loads(result[0]))
    cids.append(result[1])
    
print("ress: ", len(ress))
print("cids: ", len(cids))


for i, res in enumerate(ress):
    # added None after keyerrors:
    _typ = res.get("type")
    if _typ in ["keychange", "profile-change", None ]:
        notconvo.append({"i":i, "type":_typ, "keys":res.keys()})
        continue
    conversation_id = res["conversationId"]

print("notconvo: ", len(notconvo))
i=0
for n in notconvo:
    if n['type'] == None:
        i += 1
print("notconvo[type=None]: ", i)
        
for i, n in enumerate(notconvo):
    if n['type'] == None:
        print(f"{i}:{i+9}:")
        pprint(notconvo[i:i+9])
        break

In my particular test environment, these are the resulting outputs:

ress:  10728
cids:  10728
notconvo:  1921
notconvo[type=None]:  1834
76:85:
[{'i': 8424,
  'keys': dict_keys(['attachments', 'sendStateByConversationId', 'unidentifiedDeliveries', 'bodyRanges', 'contact', 'decrypted_at', 'errors', 'flags', 'preview', 'requiredProtocolVersion', 'supportedVersionAtReceive']),
  'type': None},
 {'i': 8426,
  'keys': dict_keys(['attachments', 'sendStateByConversationId', 'unidentifiedDeliveries', 'bodyRanges', 'contact', 'decrypted_at', 'errors', 'flags', 'preview', 'requiredProtocolVersion', 'supportedVersionAtReceive', 'editHistory', 'editMessageTimestamp', 'editMessageReceivedAt', 'editMessageReceivedAtMs']),
  'type': None},
 {'i': 8427,
  'keys': dict_keys(['attachments', 'sendStateByConversationId', 'unidentifiedDeliveries', 'bodyRanges', 'contact', 'decrypted_at', 'errors', 'flags', 'preview', 'requiredProtocolVersion', 'supportedVersionAtReceive', 'editHistory', 'editMessageTimestamp', 'editMessageReceivedAt', 'editMessageReceivedAtMs']),
  'type': None},
 {'i': 8428,
  'keys': dict_keys(['attachments', 'sendStateByConversationId', 'unidentifiedDeliveries', 'bodyRanges', 'contact', 'decrypted_at', 'errors', 'flags', 'preview', 'requiredProtocolVersion', 'supportedVersionAtReceive']),
  'type': None},
 {'i': 8430,
  'keys': dict_keys(['attachments', 'sendStateByConversationId', 'unidentifiedDeliveries', 'bodyRanges', 'contact', 'decrypted_at', 'errors', 'flags', 'preview', 'requiredProtocolVersion', 'supportedVersionAtReceive']),
  'type': None},
 {'i': 8434,
  'keys': dict_keys(['attachments', 'sendStateByConversationId', 'unidentifiedDeliveries', 'bodyRanges', 'contact', 'decrypted_at', 'errors', 'flags', 'preview', 'requiredProtocolVersion', 'supportedVersionAtReceive', 'quote']),
  'type': None},
 {'i': 8435,
  'keys': dict_keys(['attachments', 'sendStateByConversationId', 'unidentifiedDeliveries', 'bodyRanges', 'contact', 'decrypted_at', 'errors', 'flags', 'preview', 'requiredProtocolVersion', 'supportedVersionAtReceive']),
  'type': None},
 {'i': 8436,
  'keys': dict_keys(['attachments', 'sendStateByConversationId', 'unidentifiedDeliveries', 'bodyRanges', 'contact', 'decrypted_at', 'errors', 'flags', 'preview', 'requiredProtocolVersion', 'supportedVersionAtReceive']),
  'type': None},
 {'i': 8437,
  'keys': dict_keys(['attachments', 'sendStateByConversationId', 'unidentifiedDeliveries', 'bodyRanges', 'contact', 'decrypted_at', 'errors', 'flags', 'preview', 'requiredProtocolVersion', 'supportedVersionAtReceive']),
  'type': None}]

The majority of the records are of type None, without a conversationId key, but they do appear to be tracking message send/recieve/edit actions, and all seem to have a sendStateByConversationId key. This may be useful for further, future logging of these records, if desired.

In my debugging, I toyed with using python try/except blocks for the KeyError, as it could be a more general handling. These records may be assigned a "type" in the future, and handling of these kinds of errors could be more generally put in a few statements, or possibly a centralized function. Maybe for a future change.

[carderne]

Hey thanks for this. Could you adjust the PR so it's just adding the None to the list. I just updated the sqlcipher3-wheels dependency and the little comments aren't needed.