Painting with Gaussians
82 points by yogthos 8 hours ago | 14 comments
swiftcoder 2 hours ago
The effect looks decent in the foreground of some pictures, particularly the wolf and the cat. It really exaggerates the depth of field in the background, though, and those areas feel like someone ran a posterise after a blur, rather than brush strokes. Might just be a side effect of the author's predilection for bokeh, I guess. The Tokyo shot looks good throughout.
replyshen 5 hours ago
I’d suggest for the examples to pick images that are not mostly bokeh.
replyyogthos 5 hours ago
I used my own photos for the input, I find images with the subject isolated to be more visually pleasing, hence most of my photos are in this style. But there wouldn't be any difference to the algorithm if the image was full of detail as you can see with my photo of Hong Kong at the end.
replyMeteorMarc 6 hours ago
Intriguing and nice writeup. This however tries to encode all the hidden painter's knowledge. Would it be easier to generate realistic images from paintings and use these pairs to finetune an image generation model?
replyjszymborski 4 hours ago
From the article
reply> In the days of generative image models, it's still fun to see what can be achieved using traditional image transformation techniques.
> But the really fun part of the project was in combining a number of techniques, such as Perlin noise, wavelets, and edge detection, that I played around with previously in isolation. All these different tricks came together for this project, making it possible to build something greater than the sum of its parts. I find these are the most rewarding types of experiments where you can build on things you've previously learned and combine them in novel ways to make something new and unexpected. I hope you enjoyed the journey as much as I did working on the project.
yogthos 6 hours ago
You can already use genAI to do similar things. The whole point, however, was to see what I could do using traditional techniques. The other benefit is that it's something you can run locally without needing a lot of resources, and have full control over the shape of the output. You can also combine this with generative models, and have a model generate a picture, then have this repaint it. Ultimately the point was to have some fun and play around with combining these techniques.
reply
[0] https://github.com/mkaic/painterbot
It's like… the way a trained human would paint feels kind of like how MSB radix sort makes an array look more ordered, going from crude to refined. The way your algorithm builds up an i|age feels like I'm looking at a radix sort that alternates between a stable MSB and LSB radix sort, which is weird as heck. It's fascinating, really.