[Feature Request]: Better Mask Editor Canvas

[brucew4yn3rp]

Is there an existing issue for this?

• I have searched the existing issues and checked the recent builds/commits

What would your feature do ?

I know that there are a few custom nodes that address this functionality, but I am wondering if you plan to implement more advanced features when masking an image natively in Comfy.

Currently the native Mask Editor only allows for single color masking (black/white) and single mask / image layer.

Ideally, the interface would support drawing on the image itself, on a separate layer, in addition to covering the area with a mask. The drawing brush would support all RGB colors and there would be a color match eyedropper tool as well to select colors to draw with from the image.

This would greatly help guide the image generation when the mask is set to a lower opacity.

Proposed workflow

1. Open MaskEditor
2. Draw guiding sketch on top of image
3. Mask off sketched area
4. Profit

Additional information

No response

┆Issue is synchronized with this Notion page by Unito

[christian-byrne]

Have you gotten a chance to try the new mask editor?

[brucew4yn3rp]

Are you referring to this interface?

It is definitely more advanced than the old mask editor, but unless I'm missing something, I think it still doesn't have the drawing feature I alluded to in the original post.

[LukeG89]

Mask editor is supposed to only make masks, for drawing there should be a different editor, like the iTools Paint Node

[brucew4yn3rp]

Thanks, @LukeG89. I know that there are a variety of node packs that allow painting on an image, but I was wondering if something could / would be built natively in Comfy to support that functionality.

Since the MaskEditor interface already maintains the core structure, and people typically would only draw on images before masking off the area to sample, the MaskEditor seemed like a natural, one-stop place for that to live.

Additionally, if someone drew using a different image editing node, and then connected it to a load image node for masking, they’d have to first queue the workflow in some way to get the drawn image into the load image node to have the updated file for masking.

Therefore I do still believe that the best place for adding a drawing UI would be within the MaskEditor (maybe the name would be updated to something like Image Canvas).

[LukeG89]

Yeah, "Mask Editor" could become "Image Editor", and that would be pretty convenient instead of using external software or custom nodes.

But I guess it's a big change that someone has to work on, like what @trsommer did when implementing the new Mask Editor.

[brucew4yn3rp]

@LukeG89 What do you think of something like the below?

1. Add a New Tool Mode for Painting

enum Tools {
  Pen = 'pen',
  Eraser = 'eraser',
  PaintBucket = 'paintBucket',
  ColorSelect = 'colorSelect',
  Paint = 'paint'  // New paintbrush mode
}

2. Add a Color Selector in UI

const colorPicker = document.createElement('input')
colorPicker.type = 'color'
colorPicker.id = 'maskEditor_colorPicker'
colorPicker.value = '#ff0000'  // Default red
colorPicker.addEventListener('input', (event) => {
  const color = (event.target as HTMLInputElement).value
  messageBroker.publish('setPaintColor', color)
})
document.getElementById('maskEditor_sidePanelBrushSettings')?.appendChild(colorPicker)

3. Modify BrushTool to Support Painting

Add this property inside BrushTool:

private paintColor: string = '#ff0000'  // Default color

Subscribe to the color picker updates:

this.messageBroker.subscribe('setPaintColor', (color: string) => {
  this.paintColor = color
})

Detect When to Paint Instead of Mask

let currentTool = await this.messageBroker.pull('currentTool')

if (currentTool === Tools.Paint) {
  this.maskCtx = await this.messageBroker.pull('imageCtx')  // Use imageCanvas for painting
} else {
  this.maskCtx = await this.messageBroker.pull('maskCtx')  // Use maskCanvas for masking
}

Apply Paint Color Instead of Transparency

const currentTool = await this.messageBroker.pull('currentTool')

if (currentTool === Tools.Paint) {
  // Use selected paint color for strokes
  maskCtx.fillStyle = this.paintColor
} else {
  // Use grayscale for mask layer
  maskCtx.fillStyle = `rgba(0, 0, 0, ${opacity})`
}

4. Ensure Undo/Redo Tracks Painting Separately

const imageCanvas = await this.messageBroker.pull('imgCanvas')
const maskCanvas = await this.messageBroker.pull('maskCanvas')

const imageState = imageCanvas.getContext('2d')!.getImageData(0, 0, imageCanvas.width, imageCanvas.height)
const maskState = maskCanvas.getContext('2d')!.getImageData(0, 0, maskCanvas.width, maskCanvas.height)

this.states.push({ imageState, maskState })

Modify restoreState to restore both layers properly:

const { imageState, maskState } = state
this.imageCtx.putImageData(imageState, 0, 0)
this.maskCtx.putImageData(maskState, 0, 0)

[LukeG89]

To be honest, I don't know anything about this, so I can't say 😅

You can consider making a PR

[webfiltered]

@brucew4yn3rp If you would like to test out generated code, it is fairly easy to do so by simply checking out the repo and compiling the code. If you are not a developer, I would recommend getting some experience in on a smaller task than the mask editor, however.

[brucew4yn3rp]

@webfiltered Fair enough: #2921