SWIFT-1130 Suggestion for change to JSON dates for toRelaxedExtendedJSON()

[NotesClef]

I would like to suggest a change to the date format resulting from toRelaxedExtendedJSON() from Date+BSONValue.swift

In the context of relaxed, users being provided JSON find it odd to see dates represented as objects (which is more mongo style than typical JSON users we supply JSON to are accustomed):

"date_valid_from": {
            "$date": "2021-12-26T00:00:00Z"
          }

For relaxed formatting, it would seem preferable to return dates instead as:

"date_valid_from": "2021-12-26T00:00:00Z"

When we wish to return JSON preserving mongo formatting, we still have canonical.

Would this suggestion be agreeable, or is there a reason I'm missing for the dates being formatted this way for relaxed?

Thank you for your consideration.

[kmahar]

Hi @NotesClef! Thanks for reaching out.

The formatting choice here is dictated by the extended JSON specification which defines the "relaxed" date format to be identical to the canonical one.

That said, I can see how the current behavior may be unexpected and more difficult to work with.

Could you share a little bit more about your use case for converting Dates to extended JSON and how you encountered this behavior? Are you using the ExtendedJSONEncoder on a custom type, or converting a BSONDocument to extended JSON? Or using this method directly for something?

Depending on how you are running into this, there may be a workaround, or we may be able to consider making the representation configurable.

[NotesClef]

Thanks for your response!

I'm converting from BSONDocument to JSON. When I supply the resulting JSON, users have indicated that they found the formatting of the dates odd compared to JSON they are used to seeing.

Many times database I see will have dates as strings, so if converted directly to mongoldb as a string wouldn't be an issue. But I'm using Swift codable structs to handle things, and the dates are therefore converted to mongo db in date format.

My workaround for the moment is that I've forked the project so that I could modify the behaviour of the ExtendedJSONEncoder. However, I wondered if this were behaviour that others would want for relaxed JSON. But as you mentioned, I think it would be great to have the date formatting in this instance be configurable.

[kmahar]

Thanks for the info! I'm going to discuss this further with my team this week.

Another thing to mention that could provide an avenue for this -- we have considered in the past making BSONDocument encodable with the normal Swift JSONEncoder and having that produce a more plain JSON representation, where everything is represented with a number or string. While relaxed extended JSON is "relaxed", it is designed to still preserve some type information (e.g. by making it obvious that a date is date and not just a string), but I imagine many use cases don't require that level of detail.

In the meantime - do you convert from Codable structs to BSONDocument using BSONEncoder before you serialize to extended JSON?

If so, you could use the dateEncodingStrategy setting on BSONEncoder to customize the representation used for Dates when encoding to BSON, so that a string is stored. Then, when you convert to extended JSON, the dates will also just be plain strings.

For example, to use ISO-8601 formatted strings:

struct MyStruct: Codable {
    let d: Date
}

let encoder = BSONEncoder()
encoder.dateEncodingStrategy = .iso8601

let s = MyStruct(d: Date())
let doc = try encoder.encode(s)
print(doc.toExtendedJSONString()) // prints {"d":"2021-11-08T02:29:23Z"}

You can also supply a DateFormatter of choice, or specify a custom function for the encoder to call when encoding. (See API docs here). The BSONDecoder supports similar strategies for decoding as well.

[NotesClef]

Thank you again for your reply! I really appreciate your feedback.

The suggestion of using the date encoding strategy from BSONEncoder is a good one to keep in mind, and I can use that when going from a codable struct. However, this would be non-optimal for my current use case that spurred my original request.

The flow is that we go from Codable structs in Swift to generate the Mongo DB collection records. We do want to preserve the dates as Mongo date format (which allows for efficient indexes and searches by date range). It also allows for encoding back to Swift structures handling dates in a standard way (when I need to work with data in Swift).

But in the case that spurred my original suggestion, I'm retrieving data from an aggregate search - so getting BSONDocuments from Mongo. I've found that encoding back into Swift structs, and then to JSON, seems substantially slower performance than using extended JSON to return results directly from the BSONDocuments.

In many instances, for example when returning results to a different microservice, we don't need to encode back to Swift structs (if we don't need to manipulate the data in Swift) - but rather just want to return the results from Mongo as fast as possible. So in some cases it seems preferable to retrieve the BSONDocuments from Mongo, manipulate them directly, and return the JSON - skipping the step of encoding to a Swift struct (which appears to add some significant overhead, especially with larger and more complex structs).

[kmahar]

@NotesClef Thanks for the additional information! Yeah, it makes sense that going from BSONDocument - > Swift type -> JSON would slow things down substantially.

I have one more question. Does the extended JSON format (as compared to normal JSON) provide any value for your use case? I see the options here as being:

1. BSONDocument could support being encoded with JSONEncoder which would produce a normal JSON representation of it (without round tripping through a Codable struct)
2. ExtendedJSONEncoder could support per-type configuration e.g. for Dates, and produce output where some types use the extended JSON formats and some types use a custom format (like just a string for Date)

If there is value for you in still having some types encoded as extJSON, option 2 might be better, but if "plain" JSON would suffice we could do option 1.

[NotesClef]

For my current use case, option 1 would work well.

However, I could imagine for other instances where having control over per-type configuration could be helpful (as per option 2).