NodeBox is paradoxical. The design-time UI has rich interactivity; you can zoom and pan in both the view and network panes to your heart's content. But it is difficult to interact in the same way with the things you make in NodeBox.

I am particularly interested in allowing users to drag over my creations and have them react. In fact I'd like to start a NodeBox file running (playing), put it in fullscreen, hand it to an end user, and watch her interact with it. You can kinda sorta *almost* do this in several (somewhat bizarre) ways:

- Drive an animation with frame nodes. The user can then scrub the frame counter below the viewer pane to simulate a limited drag animation. Can't do this in fullscreen.

- Use the experimental mouse node. Sadly, this node detects position but cannot tell if user is pressing down on the mouse (as far as I can tell). With some ingenuity it might be possible to initiate a drag by simply hovering over an area for a fixed amount of time but this is both tricky to implement and awkward for the user. Even sadder: mouse doesn't work at all in fullscreen.

- Use the freehand tool. Unlike the mouse node, it can recognize when the user is pressing down on the mouse. It only adds to its path when the user drags. By colorizing with a transparent fill and doing lookups on the most recent point in the path it is actually possible to drag things with the freehand tool. But you will create an enormous ever-growing invisible path in the process which is bound to cause trouble sooner or later. And it doesn't work in fullscreen.

- Use the NodeBox UI. Create a small rectangle and rename the node "Cursor". Hit play and then select the Cursor node in the network pane. In the viewer pane, move the real cursor over this fake cursor and drag it to reposition the rectangle. By continuously detecting and responding to the position of the fake cursor you can drag other elements around. But "picking up the cursor" is a strange interaction for end users. And, like all of these techniques, it doesn't work in fullscreen.

So near and yet so far.

I realize that NodeBox was originally designed for output, not input. But in my work I see a steady progression from GUI to NUI, from static visualizations to animated visualizations to interactives to multi-touch interactive to "living systems" where objects exhibit complex behaviors and obey physical laws when responding to user interactions. I can only go part way on this journey with NodeBox. But with just a few tweaks, NodeBox could be as powerful at input as it is at output.

A few questions for the staff:

Are there any plans to make a mouse node that can report the state of the mouse button (via a lookup) so I could implement drags? Is there already some (possibly device dependent) Python code I could plug in to do this?

Why doesn't mousing and freehanding work in fullscreen? Is this just a bug or a more fundamental design limitation?

Do you have a roadmap for allowing fullscreen "runtime" interactions? Multi-touch interactions?

Thanks for listening,

John