Image Node !
The ability to incorporate images into the pure vector world of NodeBox is probably the most often requested feature. My image node does not provide a true image object, but does the next best thing. It makes it easy to import images or portions of images as groups of "pixels" (colored rects) which you can then work with as you would any other shapes.
My image node is free for use without restrictions.
Attached is a folder containing:
- a nodebox file with the image node and other useful nodes
- image.py, the python custom code needed by the image node
- a subfolder with a few sample images to get you started.
The image node returns a pixel group (a group of colored square rects) based on an image file. I limit the total number of “pixels” to about 100K to avoid accidentally crashing NodeBox. The whole point of the image node is to get the best image you can from the fewest Nodebox pixels. You can do this by either by downsampling (sampling a subset of pixels from the original image) or by reducing the total area using a mask.
The image node takes seven inputs:
- File - a local PNG, JPG, BMP, TIFF or perhaps other formats.
- Mask (optional) - a closed shape or a list of resampled paths (e.g. concentric circles) used to limit the number of pixels
- Image Quality - seven levels ranging from 1 to 100 kilo-pixels. The higher the quality, the more sluggish Nodebox becomes
- Image Scale - the size of the displayed image as a percentage of the original image size
- Image Position - used to adjust the position of the image on the Nodebox canvas
- Output - one of three possible settings:
- Image - return the image at the current settings
- Mask - return the masked portion of the image at the highest possible quality
- Placement - shows the image dimmed and the outline of the mask, used to adjust placement of the mask
- Optimize for PNG - if checked forces pixel size to an integer value to avoid interference patterns which would occur if the image is exported as a PNG or movie. Usually causes a slight change in quality and image dimensions.
ABOUT MASKS
There are two kinds of masks: contiguous (a single closed shape), and discontiguous (a list of resampled paths that provide individual positions to sample from the original image). I provide six examples of masks for you to play with:
- a line with 100 points
- a rect
- a capital A
- a star
- a set of concentric circles. Use the subnetwork to control the number of rings, ring separation, point separation within each ring, etc.
- a set of angled hatching lines. Connect either a rectangular or circular bound and adjust the angle, line separation, etc.
You can easily switch between each one by setting the index on the switch from 0 to 5
The image node will always try to output the maximum quality of pixels within a mask when in mask mode, regardless of image quality settings. Optimizing for PNG will affect contiguous masks (examples 1 - 3), but not discontiguous masks (examples 0, 4 and 5).
Put the image node in Placement mode when adjusting the position of your mask relative to the current scale of the image. If you set quality to low you will be able to reposition the mask more quickly; this will not affect the output of the mask. For precise placement, increase the image quality. When placing a mask you can either keep the image fixed and move the mask feeding into the image node, or keep the mask fixed and move the image beneath it using the image position control.
FILTER AND STAT NODES
I also provide a few nodes that alter the output of an image node. The add_stats node simply displays a few basic stats beneath the output of an image node (dimensions, pixels, and pixel size). This is handy when adjusting the settings of the image node.
The pixel_dots node convert the square pixels of an image group into dots. You can either have uniform dots colored as the square pixels were, or have the dots sized dynamically based on the intensity of each color (its luma value). For dynamically sized dots you can keep the original colors, or use a supplied monocolor. You can also adjust the relative size of the dots; this can make the overall image appear darker or lighter.
I also show an example of how to surround the output of a contiguous mask converted to dots with the original outline of the mask, which often improves the overall look.
I can imagine many more possible filter nodes. You could easily make a node to adjust the hue, saturation, brightness, and alpha of all the pixels in a pixel group. You could convert the square pixels to other shapes like line segments, CMYK dots, or even ASCII art. You could convert color to grayscale or invert colors. You could map the original range of colors to a more limited color palette. And you could hook any of these filter nodes to a frame node to create interesting animations.
HAVE FUN
That should be enough to get you started. Please play with the options I have provided and point the image node at your own image files. If something breaks please let me know ASAP. If you have any questions, please ask them; I realize this is a complex node that will take some getting used to and I want to document it as clearly as possible.
I also look forward to seeing any creations you make with the image node, and any new masks or filter nodes you come up with.
Go forth and create!
John
-
Image_Node_Screenshot.png
468 KB
- Image_Node_V1.zip 3.72 MB
Image_Node_Screenshot.png
Keyboard shortcuts
Generic
| ? | Show this help |
|---|---|
| ESC | Blurs the current field |
Comment Form
| r | Focus the comment reply box |
|---|---|
| ^ + ↩ | Submit the comment |
You can use Command ⌘ instead of Control ^ on Mac
Support Staff 1 Posted by Frederik De Ble... on 21 Jan, 2021 09:27 PM
That's crazy! Nice work John.
2 Posted by Alexander Gogl on 21 Jan, 2021 10:22 PM
Great work John! I really like how easy it is to clip and posterize an image :)
3 Posted by Alexander Gogl on 21 Jan, 2021 10:45 PM
Each pixel / dot can be manipulated :D
Support Staff 4 Posted by john on 22 Jan, 2021 11:01 PM
Thanks, Frederik! Thanks, Alexander!
You're right: with the image node each pixel / dot can be manipulated. This opens some new capabilities I haven't seen with other tools.
NodeBox will never be a photo manipulation tool. If you want to apply artsy filters, better to do that in Photoshop or whatever, then import into NodeBox afterwards.
But if you want to roll your own filters and act on things one pixel at a time I don't think you could do that in Photoshop or Illustrator or any other tool I've come across. You could do it programmatically in normal languages, but NodeBox makes it much easier to see what you're doing, with immediate feedback, and experiment at will. You could prototype new ideas in NodeBox and then scale them up in C or whatever.
And that's just one possible use case. I see many, many more. At the very least there is much fun to be had.
Please consider sharing some of your experiments with the image node. I am really curious to see what other people will do with it.
John
5 Posted by florisdejonge on 27 Jan, 2021 05:44 PM
Hi John,
Thanks for providing the Image Node. It’s amazing and impressive! I was still trying to use your earlier mosaic sorter (based on this thread I guess?) but this is way more user friendly.
As you suggest it is possible to convert the pixels to line segments. So I’ve attached an example of this on which I was working earlier (as discussed in our previous conversation; code libraries are in the Node Library, except rgb.py) and tried to apply this in your example. Attached are the examples. It generates points based on the brightness of each pixel and connects these in a line with every row. Works great (except for masks), especially since one can easily test a part of the image in low resolution.
Kind regards
Floris
Support Staff 6 Posted by john on 27 Jan, 2021 10:35 PM
Floris,
This is excellent! Just the kind of filtering I hoped people would experiment with. You are particularly well-suited for this because of your experience making things plotter-friendly.
Your approach relies on organizing pixels into rows. That should also work with contiguous masks (based on a closed shape like a rectangle or ellipse) since my Image node organizes those into rows. I would think you might also get it to work with discontiguous masks. Concentric circles would be tricky (maybe using small arcs instead of line segments), but I would think the hatching mask I supplied would be ideal for your approach. It would do what you are doing now, but allow you use rows tilted to any angle.
I wonder if you could mimic engravings or fine stippling work by dividing an image into multiple regions and using different filters on each region. This would require precise masks which in turn would be hard to define without some form of edge detection. Could be quite an art form in the right hands.
I look forward to seeing more of your experiments. It would be especially nice to see a video of your plotter in action on Instagram (or here in the forum for that matter).
Thanks - you made my day!
John
7 Posted by florisdejonge on 28 Jan, 2021 08:39 PM
John,
Thank you for your reply and encouragement. I would indeed expect that the connected-line-segments-filter (for lack of a better name) should work with masks. But although I take a rectangle mask, it messes up the placement of the pointclusters (see attached screenshot). I am not sure whether I connected this filter (ie network of nodes) at the right part in the existing network though.
Nevertheless, I was able to create a CMYK version of the filter. Example is attached (I didn’t dare to increase the resolution). I also added a cutout (with the mask node) for the highlights.
I will look into your other suggestions.
Kind regards,
Floris
Support Staff 8 Posted by john on 29 Jan, 2021 12:22 AM
Hi Floiris,
I didn't completely analyze your convertPoints node, but I see the basic problem. See screenshot and attached code.
Your rows of connected line segments appear to be correctly constructed, but they are not positioned correctly in the final step.
First, some background about how the image node works...
When the image node is in image mode, it outputs a group of adjacent rects, one rect for each pixel. They are ordered left to right top to bottom, but are all in one big list. You could regroup them into columns if you wanted by grouping all centroids with the same X coordinate - which it looks like you are doing in your node.
When the image node is in mask mode and the mask is a single closed shape, like a rect or an ellipse, the output is the same: one big list of adjacent pixels in left to right top to bottom order. Here again you could regroup them into columns by looking at the X coordinates, but if the mask was not a rect, the columns might not all have the same number of rows and might not all be centered on the same axis.
When the image node is in mask mode and the mask is a discontiguous set of paths, like concentric circles or the angled lines output from the hatch node, things are a little different. Instead of a single group of pixels, the image node outputs a list of groups, one for each path of the original mask, each one holding the pixels in that path. I did this so that it would be possible to operate on the pixels circle by circle or line by line.
If you suck in all the paths at once (as a list) and then ungroup them, as you do at the top of your node, you have one big list of pixels, but they are NOT in left to right top to bottom order. ALSO, the pixels are not perfectly adjacent to each other; they may partially overlap and there may be gaps in places around each pixel.
Discontiguous masks are tricky because the image node automatically tries to size the pixels to roughly fill the space, so if you space out the lines in the hatching mask the pixels will get bigger to compensate. This is why I provided a dot scale option in my pixel_dots filter, so you could adjust the overall density by adjusting pixel size. You will probably need a similar option on your node (in addition to maximum points node which controls how complex each "pixel network" is).
I did come up with a temporary workaround. I left your node as is but then simply rearranged and restocked the output. The result is not bad, but it does require knowing what angle the hatching node used (in this case, 45 degrees). Also the stacking required some experimentation to find the right overlap.
I hope this is enough information for you to adjust your algorithm to make a generic node that will work with any mask.
Please let me know if this makes sense and if you need additional help. I don't fully understand your algorithm, but I like the results.
John
Support Staff 9 Posted by john on 29 Jan, 2021 05:37 AM
Hi again Floris,
I wasn't happy with the partial answer I gave you above, so kept fussing with your node until I finally fixed it (more or less). In a burst of premature optimism I renamed it "plot".
It now works with images, discontiguous masks (including hatches of different regions at different angles), and simple contiguous masks (like a small rectangle).
Because contiguous masks always maximize quality (and therefor pixels), they quickly get too big to handle, especially if you allow a high number of maximum points for your pixel networks. And they return a single path which does not work with concave shapes. I tried fixing this problem with the mask node we worked on together, but it's just too slow. I will keep exploring this problem.
So for now you should probably just use the plot node with images and discontiguous masks.
In the process of fiddling with your node I greatly simplified it but also added additional controls. The main change was simply to read in the image node as value instead of a list, so it deals with one hatching line or concentric circle at a time. It no longer bothers with figuring out rows and columns; it just joins the density networks directly and calls it a day.
I also simplified the density networks. Now they are just connected points instead of a group of small rectangles. I think that's a cleaner solution, but you may want to go back to your way or experiment with other ideas.
Your node had a fixed margin of 2 + pixel width. This does not work well when pixel width is 1 and the pixels are adjacent. So I added a control which scales the pixel width; 100 leaves no margin, 150 creates a 50% margin, 90 create a negative margin (which helps in some cases).
I also added a checkbox to "randomize" the output. This just hooks to the random seed used to scatter points for your pixel networks. With the checkbox off this uses the same seed for each path fed to the node (e.g. each hatching line). This results in a very clean plot with parallel paths having similar shapes. Turning on randomize generates a different seed for each pixel of each path by multiplying the X and Y values of its centroids and using that as a seed. The resulting image is much noisier but perhaps more interesting.
Finally, I added line thickness, which simply sets the stroke width of the line to one tenth of the value supplied (minimum stroke width in Nodebox is .1). This doesn't effect the shape of the plot lines, but is needed to display tightly packed pixels within Nodebox. Unless you reduce line width to the minimum when plotting a rectangular mask, you will see nothing but a big black smudge.
(I don't know much about plotters, but I assume you can attach anything from a fine pencil to a fat marking pen to achieve different line widths on paper. The line width you set in Nodebox will probably predict how fine a line you will need on your plotter.)
The attached demo shows a direct image and a variety of different masks. In each case I show the output of the image node next to the output of the plot node. It may take 10 or 15 seconds to render everything; when experimenting you may wish to just render a particular plot node.
Please try fiddling with the settings on various plot nodes to get a feel for how they affect the output. The settings give you a lot of control - maybe too much control. I would be interested to hear if you think they are sufficient to get the quality you want and you feel they could be simplified.
And of course there may be things about plotting I don't understand that require additional modifications to this node. And there are many variations on how to create "density networks" to convey luma; you may already have some ideas in mind.
So please give this a spin and report back. And if you can, post a video of the plot node in action on your plotter! A solid plot node would be a nice gift to the NodeBox community.
Thanks,
John
10 Posted by pyfave on 29 Jan, 2021 11:00 PM
This is great. Thank You !
11 Posted by florisdejonge on 30 Jan, 2021 01:28 PM
John,
Thanks again. It’s great to see how you’re able to take something and make into this full-fledged tool.
A couple of questions about your use of Nodebox, since I really like how you take all controls to the root level and making things user friendly
* How do you make these drop-down menus for choosing options? For example the ‘image quality’ and ‘output’ of the image node itself? * To get a checkbox you just add a Boolean-node in the subnetwork, is that correct? * Is there a way to add the dependent code libraries to a folder to post it on the forum, without having duplicates in a lot of folders? Of point nodebox to a standard folder?
Some remarks regarding your improvements:
* The margin was indeed meant to make the pixels slightly overlap, to make gaps between rows and columns of points disappear and makes the image more seamless. To make this a percentage of the pixel size is a good idea. As far as I understand your network, the ‘point separation’ in the hatching-node and ‘concentric’ node does the same, correct? It feels like I am doing the same with these two functions. * I personally like the randomness of each arrangement of points, since this noise give more variance to the image. * I consider the step to remove the rectangles and connect the points direct and then join these clusters the biggest improvement. * When applied with a plotter the strokewidth is just used to give an impression of the end result, which is dependent on the scale of the image and the thickness of the pen or marker used.
Regarding ‘plot4’ (which is most similar to my earlier attempt): Right now the plot-node converts the pixels into points (the darker the pixel, the more points), which are then all connected. Although this is plotter-friendly (no up and down movements), this leads to these diagonal lines from the end of the row to the beginners, which interferes with the image in my opinion (first image on screenshot).
Therefore, I only connected the points per row, see example attached (second image). Another option would be to reverse the number of points per row as well as the arrangement of the amount of points per pixel (to make a zigzag motion), but I think that would be overly complex (with the loss of speed for travel time now probably being negligible).
In regard to your example called ‘plot’ and ‘plot1: these connections between every rotated row with the circular and rectangular mask doesn’t seem to be there, but I can’t work out why exactly. Probably because of the hatching-node? So, instead of my solution above it seems also possible to just make the rectangle which is fed to the bounding shape of the hatching-node the same size as the original image (third image). Which would perhaps make the plot-network more clear.
Enough testing the network for now, I guess. I’ll try to actually plot something with it :)
Kind regards,
Floris
Support Staff 12 Posted by john on 30 Jan, 2021 09:19 PM
Floris,
I'll answer your NodeBox questions first, then tackle the Plotting questions in a separate comment.
To make a subnetwork with a menu port:
Inside your subnetwork you can then turn that string into a number if you want (or not). I usually define keys of 0, 1, 2... and then feed it directly into a switch node.
It's pretty nice. All that power hidden away in the Metadata dialog. And even though the port now functions as a user-friendly menu, you can still feed it key values directly from another node. My image node has seven levels of quality, but you can also just feed it the number of kilo-pixels directly from another node.
Yes, to add a checkbox just publish a boolean node.
I wish there was a way to automatically include code libraries in the .ndbx file so we wouldn't all have to muck around with adding them to folders and pointing to them from the File menu. It's a common feature request. But for now we'll have to muddle through. Even if you could add the libraries into your own copy of Nodebox, your code would break when you publish it on the forum and other people try to use it.
I keep all my Python and Clojure code files in my node library folder. Whenever I create a new project that needs any of them, I make a folder and drag copies into that folder. Whenever I share my work on the forum, I zip the whole folder so that people don't have to repeat that process. It's a minor nuisance, which is why I go to great lengths to do everything in Nodebox. But sometimes, as with the image node, the pain is worth it.
John
Support Staff 13 Posted by john on 31 Jan, 2021 12:08 AM
Floris,
Yes, you are right that the point and line separations in the hatching node has an effect similar to the controls on the plot node. It's somewhat redundant and when I was playing with them I went around in circles achieving similar effects in different ways.
I also noticed the problem with the diagonal lines connecting the end of each row to the beginning of the next row. It's easy to fix in several different ways (as you did in your plot4 node). The reason there are no such lines when using the hatch or concentric nodes is that they are already segregated into separate "rows". They feed a list of separate rows to the image node, and the image node preserves that by spitting out a list of separate pixel groups (when in mask mode for discontiguous masks). The plot node operates on one group at a time, so also outputs separate groups with no connecting lines between them.
The end result of all this is that we now have many options for creating a clean plot. You can either put the image node in image mode and use the plot node to fine tune the output, OR control density and direction of pixel "rows" with the hatching node, feed that into the image node in mask mode, and apply additional adjustments (or not) using the plot node.
If you go the image mode route, you could correct the diagonal lines using a separate plot node for that purpose (as you did with plot4), or make a post-processing node that removes those diagonal lines (easy to spot since they are WAY longer than any of the other line segments except maybe in weird corner cases), or maybe try to make a clever combined plot node that automatically detects and removes or avoids those diagonals only when needed.
(I could also eliminate this problem in the image node in image mode by automatically reversing the direction of every other row so that there would only be shot lines between pixels at the beginning or end of rows if that image is ever plotted. Or maybe just output images as groups of horizontal rows instead of as one giant group.)
But maybe, if you're going to do an actual plot, you might want to use the hatching node to create a discontiguous mask anyway - in which case the diagonal lines are no longer an issue. In that case you might not need to have essentially redundant controls in both the hatching and plot nodes.
One other difference between plotting images (image mode) and plotting masks (mask mode) is that in image mode you can set an arbitrary level of quality. In mask mode, quality will always be maximized. When using masks you control the quality by setting the line and point separations.
I feel like we haven't quite landed on the perfect solution yet and need more experience to judge how these three nodes (hatching, image, and plot) should best work together. This is where you come in. Start actually using them to create real plots. You will probably gravitate to a preferred method. Based on that maybe we can revisit the plot node and perhaps simplify it a bit more.
Thanks for collaborating with me on this. I look forward to seeing a video of the plot node in action!
John
14 Posted by florisdejonge on 01 Feb, 2021 07:17 PM
John,
Thank you for you expanding on your use of Nodebox. I was already wondering whether it would be possible to render the image quality somewhere between ‘clearer but less responsive’ and ‘medium clarity’. So, it’s good to know where it can be changed, but also how it’s made. I also noticed that the image quality didn’t change when using the hatching node, so thanks for clearing that up.
The linking of the right code libraries is indeed a minor nuisance. Similar to (extensively) copying and pasting nodes from the Node-Library 2-2 ;). It would be great if these would just be part of the software. So hopefully, someday, they will.
As far as I’m concerned it’s not a problem that the plot-node generates discontinuous paths. I did made two drawings with the plot-node. Results can be found here (and a short videoclip as well; I want to try a timelapse in the near future). I still need to figure out the balance between detail, scale and density. I made some minor adjustments to the plot-network before using it.
I considered the random positions of the points a litte too much. So I added a ‘wiggle’-node instead, after the scatter node, so I could control the amount of randomness. I also switched the seed-port of the wiggle-node with the seed-port of the scatter-node. In regard to the seed, I saw you’ve generated unique numbers to feed the seed-port by multiplying the x and y position. But this somehow lead to the pointclusters on the horizontal row and the vertical row to be the same if the pixel value was also constant. This is especially visible in lower quality. I therefore added a shuffle node.
I will report back when I’ve discovered more about the application of the plot-node. Maybe a dedicated discussion thread is useful, so this thread can be used to add to the discussion of the use of the image-node itself. I will try to add to the discussion on the topic of dedicated plotter-nodes soon as well.
Kind regards,
Floris
Support Staff 15 Posted by john on 02 Feb, 2021 05:12 AM
Floris,
I had the same thought about bifurcating this thread.
So yes, if you or others want to talk in general about the art and special needs of plotting, please continue that discussion here (I just added some more questions):
http://support.nodebox.net/discussions/general-discussion/15034-spe...
And Floris, let's you and I (and anyone else interested) continue to work on the evolution of the plot node in particular using this new thread:
http://support.nodebox.net/discussions/show-your-work/406-plot-node
And anyone else who wants to talk about the image node, please continue adding to this thread!
Thanks,
John
16 Posted by msensoy on 06 Apr, 2023 10:51 AM
Hello John,
I have a question and I would be really glad if you can fix it. I tried this node and it's amazing. The only problem I'm facing with is when I export output as svg and open it in Illustrator, there are being so many zero size points. I've tried several illustrator scripts to clean up empty points. Is there a way to add any node inside this preset for to delete empty points?
Thank you very much.
17 Posted by florisdejonge on 06 Apr, 2023 12:17 PM
Hi msensoy,
This is perfectly doable in Nodebox itself. It's useful to keep it a separate function from the image node, since zero size points occurs more open. I usually use a lookup node on size (bounds.width or bounds.height), add a compare node and a cull node. Hope this helps.
Floris
18 Posted by msensoy on 06 Apr, 2023 01:50 PM
Sorry, I guess I'm doing all wrong here, I've attached a screenshot.
19 Posted by florisdejonge on 06 Apr, 2023 07:31 PM
The compare node is only necessary once. See attached example. It's also possible to check if either the height or the width is 0 per path, but that's a little more complicated.
Floris
Support Staff 20 Posted by john on 06 Apr, 2023 08:04 PM
Msensoy,
Floris is correct - you can fix this problem inside Nodebox before you export. I call this issue "dust", tiny particles or points or microscopic path segments that sometimes appear as residue after other operations.
Floris is also correct that you can detect them by culling out paths which fail a size test. But instead of using bounds, I use path length or area.
Using bounds is slightly messy because vertical or horizontal line segments can have 0 width but non-zero height or vice versa and you might not want to cull those. Also culling only paths with zero size will miss dust particles that have a finite but tiny size (like .0345 times ten to the minus 13).
To test path length use a lookup node with the key set to "length". Do not use the length node (which returns the number of characters in a string). And for your compare, instead of equal 0, say less than 1, that is, less than 1 pixel, or .01 or whatever limit seems reasonable in your situation.
Path length may not catch unclosed path fragments composed of multiple free-floating points. If path length doesn't work in your situation, you can use the area node from my Cartan Node Library. This node returns the area in square pixels of any polygonal path. Curved paths will return approximate area. Area is great at detecting dust. Anything with an area of less than .01 is dust. In fact anything with an area less than 1 or even 10 is probably not going to be something you want.
Just do a single size test with path length or area and cull anything below an acceptable size before you export.
If you're still having problems, send me an example project and I'll show you how to add the cull.
Thanks to Floris for helping out. That's the way this forum is supposed to work.
John