Issue

When you have a selection within the annotatable container and click outside - the selected annotation won't be dismissed.
Instead, the userSelect method is called on it:

That's because the pointerDown method is initialized only on the container, but the pointerUp is initialized on the whole document. Therefore, the currentTarget is still present when the latter is executed, and the evt.timeStamp - lastPointerDown.timeStamp is always too large: