plugins and back compat #212
Description
When we break back compat on levelup all the plugins may or may not break.
A lot of level-* modules in the ecosystem use level-sublevel to extend a LevelUP instance with new functionality.
This works great up and until we change levelup in a back compat breaking way like removing createWriteStream() or changing the createReadStream() range options API.
The core problem is that level-* modules have an invisible peer dependency on levelup at a certain version. When levelup changes those modules break.
It also means that as the ecosystem grows you won't be able to use certain level-* modules together because you have module A using the old range query API and module B using the new range query API.
Core problem
plugins do not depend on something that can be versioned statically and instead depend on something passed in at run time with a certain interface.
Possible solutions
- Never break back compat. This may sound silly but it's genuinely the solution.
- Peer depend on things. I highly recommend against this, my experience with peer deps is a disaster. @shama also has opinions about peer deps being a "solution" for plugins.
- statically require and depend on levelup. This is the solution that would work the best.
How do you require levelup in a plugin?
If we were to combine 1 & 3 together we could in theory choose to never break back compat on leveldown and break levelup up so that its a set of functions that take a leveldown instance as an argument.
The solution is to break levelup into two modules. One module is levelup, the interface you use in your app because you want the nice interface and the streams and everything. The second is a set of functions that are versioned independently and can be used by level- plugin authors
In that regard a level- plugin author would a leveldown instance (probably wrapped by levelup or passed directly) and it would call functions from the levelup module on the leveldown instance. This means that if we remove the createWriteStream from the nice levelup interface it means nothing for level- authors because they always depended on the write-stream.js module directly and called it as a function.
What does this look like for level- authors?
Imagine I had an imaginary copy-users level- plugin that would currently look like
function copyUsers(db) {
db.copyUsers = function (targetDb) {
db.createReadStream({
start: "users~",
end: "users~~"
}).write(targetDb.createWriteStream())
}
return db
}If I wanted to use the new functional equivelant it would look like
var readStream = require("levelup/fns/read-stream")
var writeStream = require("maxes-write-stream")
function CopyUsers(db) {
return function copyUsers(targetDb) {
readStream(db, { gt: "users~", lt: "users~~" })
.pipe(writeStream(targetDb))
}
}Notice how this second version uses the new gt & lt api for read stream because it uses read-stream at a specific version that has that new api. It also uses maxes write stream from npm.
Any breaking changes to the levelup interface for db do not break this module (as long as abstract leveldown doesn't change)
What does this look like for levelup
The public levelup interface doesn't need to change. The internals would be refactored so all methods re-use the functions.
Those functions could live as seperate versioned modules or as files in a folder inside levelup.
Benefits.
As long as the functions are individually versioned we can break back compat on the public db interface of levelup as much as we want without breaking anything in the level- eco system.
In the long run this means that we will never run into the "this plugin only works with levelup1 not levelup2" problem. We still run into a "this plugin only works with abstract leveldown1 not abstract leveldown2" problem.