I have created a monster
I just finished a really beautiful visualization using NodeBox that I will soon be sharing. But I have a problem…
This visualization has a large number of text paths - 904 paths with over 3.3 million points. As a result, it's right on the edge of what NodeBox can handle. NodeBox gets memory heap errors if I try saving it as an SVG. If I open, crash, and restart I can get a PDF; the PDF is viewable, but takes up to 20 seconds to refresh every time you zoom or pan.
I took this thing to my local print shop and ended up spending a whole day there. We tried one thing after another, but their system couldn't print it. A seasoned person there said in all her years she had never seen a file that did this. I even tried rasterizing it in Photoshop at 600 dpi (not ideal - the printer does 2400 and there is some blurriness in fine details at 600). It took 30 minutes to convert to a TIFF which then weighed 2.3 gigs. All no no avail - their system couldn't handle that file either.
I will try another print shop next weekend. But man o man do I wish NodeBox had text objects!
Keyboard shortcuts
Generic
| ? | Show this help |
|---|---|
| ESC | Blurs the current field |
Comment Form
| r | Focus the comment reply box |
|---|---|
| ^ + ↩ | Submit the comment |
You can use Command ⌘ instead of Control ^ on Mac
1 Posted by rioch on 21 Mar, 2017 04:47 PM
well, silly as it sounds you may try changing the font, could make some difference on a big file with lots of text... as a quick test, pointCount for a textPath: caslon > 388359, verdana > 266037, futura > 125078
Support Staff 2 Posted by john on 22 Mar, 2017 01:30 AM
Rioch,
Brilliant idea!
After some experimentation, I found that SanSerif had the best reduction. For this particular case, a column of small print, there was almost no visible difference. And this one change reduced point count by 1.2 million and the overall PDF file size by a third (16.6 megs to 11.2 megs)!
I will try going back to the print shop tomorrow. It will be interesting to see if this makes the difference.
Thank you!
John
Support Staff 3 Posted by john on 22 Mar, 2017 02:00 AM
One quick followup...
As a result of this change, I am now able to save the viz as an SVG (not that I really need to; I just couldn't before). The SVG (essentially a large XML text file) weighs in at 45.6 Megs.
It's interesting to see what will open a file this big. I am impressed that both TextEdit and Sublime Text had no trouble. Sublime Text's line index shows the final line as 23,627 - but most lines are multiple screen heights tall so I'd probably need a 50,000* foot-tall screen to see it all without scrolling.
Chrome and FireFox on the Mac were both able to open it. Both would only allow me to reduce the zoom view to 25%, which was still way too large to see more than a small chunk of the overall viz.
Safari got part way, but crapped out with an amusing error message:
error on line 8802 at column 9: internal error: Huge input lookup
All in all, definite progress!
John
*50,000 is an exaggeration. If I maximize my laptop screen, assuming an average of 120 characters per line and 47 lines per screenful, each screenful 6.5 inches tall, the display of 45,649,282 characters would be only 4384 feet tall, or about 4.45 Eiffel Towers.
Support Staff 4 Posted by john on 26 Mar, 2017 11:49 AM
Update: success, at least in part.
The reduced file still wouldn't render at the first shop, but a second shop was finally able to print it. It now hangs in my kitchen (the only free wall space in our cluttered home big enough to hold it.)
The catch is that in order to print it, the second shop had to reduce it to 600 dpi. At 52 x 39 inches, higher resolutions would take many hours to render and, went sent to the printer, would exceed the TIFF format's 4 gig limit. My print looks good overall, but some details are muddied and the fine print (which I hoped to at least see through a magnifying glass) is unreadable.
Yesterday I found a way to replace NodeBox's text paths with text objects by doing search and replace on the SVG output and then converting that file to PDF. Nodebox itself can generate a CSV file containing the needed search and replace strings and I found a utility that can automate the replacements.
But I now understand that this would probably not help me in this situation; the text would just be converted back to vectors and then rasterized anyway by the printer. My experiments showed that different systems introduce slight changes to font appearance; many professionals advocate converting all fonts to vector paths before printing in order to prevent this. So NodeBox's text paths actually offer more control when printing (as long as you don't have too much text).
I will post this visualization in the Show Your Work thread sometime soon. This has been a challenging project but I've learned a lot along the way.
5 Posted by rioch on 26 Mar, 2017 01:50 PM
That's a big poster, good to know you could at least print it, it a pity there's not better way yet. Even Nodebox Live is not exactly there from what I know, as text doesn't seem to have a width port, but if you take care of the formatting then svg exports as text objects. About outlining fonts to print, I think if you embed the font in the pdf you should usually be fine.
Manually editing the svg/pdf I was about to suggest you the other day. Or even better, do a custom svg export layer with all your text, a bit like the csv output node you did, but with svg tags. You can even flow text inside shapes with svg.
Support Staff 6 Posted by john on 30 Mar, 2017 01:48 AM
Just posted the poster with a link to more info in the Show Your Work thread:
http://support.nodebox.net/discussions/show-your-work/187-who-likes-me
John
7 Posted by rioch on 30 Mar, 2017 02:33 AM
impressive, a lot of work and many clever details, I liked it
thanks for the closeups, enjoyed reading about your process too
Support Staff 8 Posted by john on 01 Apr, 2017 12:40 AM
Thanks, Rioch.
One small discovery that may be useful for showing other large files like this...
My iPad pro is normally a great way to show "deep zoom" visualizations; you can use apps like PDF Expert to zoom and pan effortlessly. But that didn't work very well for this monster. Every time I zoomed or panned it took as long as 20 seconds to repaint and deblur. Useless.
What does work, sort of, is posting the SVG on a website and viewing it through the Chrome browser app. (Safari can't handle it.) It may take a minute to load, but once in place Chrome could zoom and pan with only momentary delays.
The one catch is that I can't zoom all the way out to see the whole poster. At most I can only see about one ninth of it. There is also a limit as to how far I can zoom in, but I was able to read the fine print at maximum zoom, which is all I needed.
So now, to show this project, I can show the overall view from a screenshot stored in Photos, then switch to Chrome to zoom in quickly to various details. I can zoom down to see the ID code in any given post bubble, then quickly swing over to the iindex and look it up.
I will be adding the Chrome IOS app and NodeBox SVG output to my toolbox from now on.
John