Issue with the color quality of the generated images #12
Comments
|
Thanks for bringing this up, I did not even notice it before! I've now added scaling options:
Usage: python app-stream.py --resize_method=basic |
|
@BinitDOX |
|
Due to the presence of CUDA, this fork works extremely fast. |
|
@BinitDOX |
|
Created a new backend, with the ESRGAN super resolution and unexpectedly time taken to colorize at 576px and upscale using super resolution is much less than just coloring at larger size. The quality is better than the original. Thank you for referencing that model! Re-clone the repository, download and place the generator weights in the networks folder of Backend-New. Then simply run: That error is because I updated the extension (frontend) files, you can fix it by re-loading the extension package into the browser.
|
|
This repository desperately needs reorganization and redocumentation. |
|
Thank you very much for your efforts! PS. BTW, why are there so many warnings in the log? |
|
Oh my bad, I forgot a comma in the Firefox-AnySite extension script. Please update ( You can ignore those warnings. The first few security warnings occur if we load the entire model (model network code + weights), can be fixed by loading the network first (state_dict), then loading the weights. The source code warnings occur if we essetially load old model with new (updated torch) code, can be fixed by resaving your own copy of the model, then you can always load this saved model. |
|
Still the issue continues to exist :( MangaColorizer-server-connection-issue-Video_2024-07-28_17-23-03.webmLog from the Firefox console |
|
In Chrome:
|
|
Sorry, my bad, it turns out that I had to click the "Colorize!" button to start colorization on the site, although on earlier versions I had colorization automatically started right after loading the page with manga. |
|
I noticed that 2 files ( |
|
And one more thing, can it be possible to make a "Show Original" button in addition to the "Colorize!" button? https://senkuro.com/manga/re-monster/chapters/119429828913939988/pages/4
|
|
|
Thanks a lot again!
|
|
The colorizer simply queries the img tags and replaces the grey images with colored. A simple way to make folder structure is to make it exactly like the URL: https://senkuro.com/manga/re-monster/chapters/119429828913939988/pages/4, so |
|
Thank you for the explanation. |
Besides manually setting the title and chapter, perhaps it's worth considering automatically generating them based on the browser tab's title? |
|
Colors are weird, probably because the model was trained by resizing the images down to 576. Then multiple Conv2D encoder blocks encodes the image down to something like 4 or 2. Then some processing (pattern learn / coloring etc) happens at this latent encoded space, then the image is decoded again by TransposeConv2D decoders back to 576. If you give the model an image size greater than 576 and if it dynamically doesnt add more Conv2D encoders, the encoded latent image stops at a higher dimention. For example if 576 goes down to 4. Then 1152 will go down only to 8. Processing at a larger size will not only take much more time but also return bad results. This could be the reason for bad colors. I have previously experimented with GANs before (~3y ago), if you want to do it too:
For example in this code: I'll consider the automatic structure generation. |
|
Oh, thank you so much for such a detailed explanation! I really appreciate it! |
|
I conducted the following experiment. Original images:
Processed images:
Eh, the color distortion is still there with even worse results. |
|
Will check this soon. |
|
This tiling is another method which could've been used instead of just upscaling., Find closest width divisible by 32, then divide the image into tiles of 576 width, then colorize, then reassemble. But since the colorizer was trained with a dataset distribution containing full images where it probably learnt about context (grass, trees, buildings, relations, etc), if you give it tiled images (different distribution), it fails. That's probably why the other fork, even though it has the code to use tiling, doesn't use it for colorizer but only for super resolution. But the tiles you provided are rich with context and should've worked.
|
|
These eye-popping colors are just terrible, it's better as it was originally, and if the text is poorly visible, then look at the black-and-white original. |
|
I'm unable to find the sigma, can you please link it? The caching configuration will have to be set for each site, although it's quite easy, it has to be hardcoded once manually. I've added it for these sites, you can update and test. Provide me a list of sites that you want it to work for. I usually only use a site like mangakakalot. |
I've already done it, but to no avail.
Thank you, it's great! |
Great idea! will do that tomorrow along with the sigma param. |
|
Configuration file created. The previous v1 project also had an option to give color hints, basically driving the model to follow those colors. Since it was manual work, it was probably removed in v2, along with params like sigma, etc. |
|
It's not that urgent, but is it possible to implement the same feature to give color hints in MC, since the eye-gouging colors in some images can be really upsetting. For example, when someone is depicted eating steak or fish, and they are green, as if they have gone bad. 
|
|
I have also noticed how often food tends to be colored green and wondered whether this is because the training data is skewed toward green food. Does our latest version let us try this page with 576 width to see if that helps? All the other coloring in this one looks great, though. |
|
@vatavian |
Since the model works best with 576, it is now fixed in the latest version, you can change and experiment with it ~line 256 in app-stream
Most probably a model problem, a denoise of 100 changes it's color, but also removes details from the image. |
I agree, it looks better in relation to food, but the fact that there are practically no details and the picture in general turns out to be blurred and the characters' faces are deformed does not give this method an advantage, because in fact you have to choose between bad and bad.
|
Right away I want to thank you for this wonderful software that allows to colorize manga on the fly.
I've known about the source app for a while now - https://github.com/qweasdd/manga-colorization-v2/, and I'm familiar with its
limitation where processing images wider than 576px results in incorrect coloring.
Is it possible to colorize the image at a size most suitable for the program, i.e. x=576px, and only after this, scale it to a larger size?
Let me explain with an example.
https://mangadex.org/chapter/d72a3eb8-2b20-4c39-a46f-d282eb05e64f/3
Original image (1125 x 1600 Pixels)
How does the image look after processing if the program processed it at dimensions of
x = 576px, as originally intended (576 x 820 Pixels)?This is what the image looks like when processed at sizes exceeding 576px in width (960 x 1367 pixels).
The text was updated successfully, but these errors were encountered: