Skip to content

objectid: After 2038, object id generation will fail. #92

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yaoqibin opened this issue Dec 19, 2018 · 6 comments
Open

objectid: After 2038, object id generation will fail. #92

yaoqibin opened this issue Dec 19, 2018 · 6 comments
Labels

Comments

@yaoqibin
Copy link

The code as follows:
def __generate(self):
"""Generate a new value for this ObjectId.
"""

    # 4 bytes current time
    oid = struct.pack(">i", int(time.time()))

Error message as follows:
packages/bson/objectid.py", line 170, in __generate
oid = struct.pack(">i", int(time.time()))
error: 'i' format requires -2147483648 <= number <= 2147483647

@yaoqibin yaoqibin changed the title objectid: After 2038, object id generation will failed. objectid: After 2038, object id generation will fail. Dec 19, 2018
@Parkayun Parkayun added the bug label Dec 19, 2018
@Parkayun
Copy link
Member

can you give me input data?

@yaoqibin
Copy link
Author

can you give me input data?

I just change the system time to 2100, then this bug ouccr.

date

Wed Jan 6 09:46:16 JST 2100

python

Python 2.7.5 (default, Aug 29 2016, 10:12:21)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import struct, time
struct.pack(">i", int(time.time()))
Traceback (most recent call last):
File "", line 1, in
struct.error: 'i' format requires -2147483648 <= number <= 2147483647

@yaoqibin
Copy link
Author

yaoqibin commented Dec 20, 2018

The max of int is 2147483647(2^31-1) and the seconds of 2038/1/19 03:14:07 UTC are 2147483647, after this time, the int is out of range.

@yaoqibin
Copy link
Author

yaoqibin commented Dec 20, 2018

We may use a unsigned long int(struct.pack(">L", int(time.time()))) to describe the time, but the object id may use extra 4 bytes.

@amcgregor
Copy link

The only "correct" way to correct this, preserving the lexicographical sort order of the IDs, is to expand the size of that field as we more closely approach the limit. This would likely require re-encoding of every ID present in the dataset, as variable width fields will not sort correctly. If you're going to expand, go whole-hog. Bump from 32- to 64-bit and have 54 zeptoseconds (~585 billion years) to worry about the next precision hike.

@wegylexy
Copy link

ObjectId("ffffffffffffffff00000000").getTimestamp() is ISODate("2106-02-07T06:28:15.000Z"), so having it unsigned is the "correct" way for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants