Add interface for simple compositing layers to dc<%>

[lexi-lambda]

@mflatt This PR is a work-in-progress, as it still needs tests and docs. However, before I write those, I wanted to ask if you think I’m on the right track. Here’s a summary of the API so far:

• We add two new methods to dc<%>:

  (make-layer) -> (is-a?/c dc<%>)
  
  (draw-layer layer [x y]) -> void?
    layer : (is-a?/c dc<%>)
    x : real? = 0
    y : real? = 0

• make-layer creates a new dc<%> that inherits its backend configuration, pen, brush, font, text foreground/background, alignment scale, and smoothing from the parent.

• After drawing to the new dc, you can draw it onto the parent dc using draw-layer, which applies the current transformation and alpha.

In additional to any general comments you might have about the API, here are the implementation details I’m most unsure about:

1. In draw-layer, I don’t do any setup except installing the right smoothing before drawing. I have no idea if I’m missing something.

2. The record-dc% implementation seems to work fine, but I’m not totally confident about it.

Finally, here’s an example program that uses layers:

#lang racket
(require racket/draw)

(define bmp (make-bitmap 400 400))
(define dc (new bitmap-dc% [bitmap bmp]))
(send dc set-background "white")
(send dc clear)
(send dc set-brush "black" 'solid)

(define layer (send dc make-layer))
(send layer draw-rectangle  50  50 250 250)
(send layer draw-rectangle 100 100 250 250)

(send dc set-alpha 0.5)
(send dc draw-layer layer)
(send dc scale 0.5 0.5)
(send dc draw-layer layer 200 200)

(send bmp save-file "/tmp/out.png" 'png)

This produces the following output:

[soegaard]

What is the right way to think of a transformation of a layer?
Is it transformed as a whole?

If so, special care is needed for brushes which has their own transformation.

A concrete example:

Let's say we have a layer in which a circle is filled with a certain checkered brush.
Now the layer is translated using a transformation.
This moves the circle - but is/should the pattern inside the circle also be translated?

[mflatt]

This API seems generally good to me.

I'm unclear on how alignment is meant to interact between the layer and draw-layer. If the offset passed to draw-layer is not aligned, then how do different alignment modes interact? Not allowing offset arguments to draw-layer and disallowing a difference in transformations/alignment may help with this question (i.e., it works to align within the layer) and @soegaard's point, but I'm not sure.

[lexi-lambda]

Originally I was thinking that if draw-layer in 'aligned mode aligns the coordinates of the layer, and drawing to the layer itself is done in 'aligned mode, then everything ought to work out. But I realize now that isn’t true: the layer would have to have the same transformation as the dc<%> it’s drawn into for that to work. If the layer’s transformation matrix were initialized to be the same as the enclosing dc<%>, then inverted in draw-layer, that would be a little closer, but it would still break down if the layer were further transformed.

I see a few ways out of this situation:

1. Completely ignore the underlying dc<%>’s transformation matrix in draw-layer and draw the layer directly into pixel-space. This way, if the layer is drawn in 'aligned mode, the results will still be aligned.

   The downside to this approach is it becomes impossible to transform a layer, even if you don’t care about alignment.

2. Give up on using Cairo recording surfaces and make all layers use record-dc%. Then alignment can be applied when the record-dc% is replayed, ensuring consistent alignment even if the layer is transformed.

   This is attractive from an API standpoint, since it ensures things “just work.” However, it would likely be less performant (though by how much I cannot say). Transferring drawing from a record-dc%-based layer onto a dc<%> would likely still need to go through an intermediate Cairo recording surface, since otherwise draw-layer would have to spend an arbitrarily-long amount of time in atomic mode.

3. Allow layers to be transformed, and document the caveat that alignment is not guaranteed if you do so. This is basically just kicking the problem to users, which gives people the most flexibility, but isn’t a very nice API.

I’m inclined to try option 2, since it seems like the nicest API by far, and it isn’t clear that the performance difference is relevant—I’m not sure dc<%>-based drawing is terribly speedy as it is. Do either of you have any thoughts?

[mflatt]

I'm ok with either 2 or 3.

I almost brought up the atomic-mode issue myself, but I think other methods already have that problem, in which case this change wouldn't make things worse. Any future, general repair to constrain atomicity to the drawing context should work for layers. Then again, if I'm wrong about existing problems, then it may be worth thinking that problem more.

[soegaard]

Would it be possible to use a layer for filling? Currently `draw` can fill a path using a brush. The current brushes are either single color, bitmaps, gradients or one of a fixed set of hatch patterns. There is currently no way of making a custom hatch pattern. As a workaround one can fill a path with a custom pict, using a clipping-region. However normally a pict doesn't repeat/extend itself so one might need to make a series of copies of the pict, before using the clipping region. Looking at `make-pattern-service` in draw/dc.rkt I think CAIRO_EXTEND_REPEAT can be used to make cairo pattens (draw layers) repeat themselves. The question came to mind after seeing https://riccardoscalco.it/textures/ a library of "textures" aka hatch patterns. Thinking of ways to implement this with `draw`, I realized that we need some kind of "infinite repeat pattern" concept. /Jens Axel Den man. 22. jun. 2020 kl. 16.32 skrev Matthew Flatt < [email protected]>:

…

I'm ok with either 2 or 3. I almost brought up the atomic-mode issue myself, but I think other methods already have that problem, in which case this change wouldn't make things worse. Any future, general repair to constrain atomicity to the drawing context should work for layers. Then again, if I'm wrong about existing problems, then it may be worth thinking that problem more. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#27 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADQXRNKDOOTBJR2XB2U6P3RX5TQFANCNFSM4OBEOYOQ> .

-- -- Jens Axel Søgaard

[soegaard]

@lexi-lambda I like your proposal and would hate to see it forgotten.
Do you remember what's needs to be done to finish it?

[mflatt]

After revisiting this, I've added different methods to dc<%> in a801d88: start-alpha and end-alpha.

The make-layer API seems more general, and therefore more attractive, at first. I went with start-alpha and end-alpha only after taking the API and implementation in this PR and making it work right with alignment: https://github.com/mflatt/draw/tree/composite2. But making record-dc% work right with that interface turns out to be really complicated — partly due to the way a recording from record-dc% is meant to adapt to a draw context's scale when it is rendered, which means there are layers of relative transformations. (The implementation still has a bug related to creating a layer in a record-dc% that is replayed on another record-dc% that is in turn rendered to a drawing context that has a non-identity transformation.) Also, even though it was using Cario's recording surfaces, it did not seem on a good performance path in the long run.

After taking a step back, I decided to try again with my first thought, which was just to use cairo_push_group and cairo_pop_group. In principle, that should give good performance by asking Cairo to do exactly the work needed for layered alpha composting. The simple experiment in https://github.com/mflatt/draw/tree/composite1 has bad performance on my machine, however. It seems that cairo_paint_with_alpha on a popped group doesn't recognize the extent of drawn content, and so it spends a lot of time compositing buffers that match the size of the target context.

The start-alpha and end-alpha here are implemented somewhat like this PR by using a Cairo recording surface, but the constrained API makes it potentially faster. The more restricted interface also avoids all sorts of alignment and relative-transformation issues. If cairo_push_group and cairo_pop_group turn out to perform well for some platforms or drawing surfaces, either now or in the future, then it's easy to switch the implementation to use them.