console graphics (was Re: NetBSD wishlist)

[David Young <dyoung%pobox.com@localhost>]

On Wed, Nov 24, 2010 at 02:42:59PM +0200, andre.dolenc%kolumbus.fi@localhost 
wrote:
> + Anything UI and graphics related (e.g., how do integrate the UI
> generated by 3D graphics and overlay on video output. Think of a
> camera UI for mobile: the UI is transparent and overlayed on the
> viewfinder which is, essentially, a video stream before the picture is
> taken); Wayland kind of window system, etc.
> Even if wayland idea is good for mobile, I am not sure the design and
> implementation is the best possible.
> If i recall, there was a GSoC that did some work exposing graphics HW
> to userland.

I've been thinking about console graphics in NetBSD because it is so
sad that the "native" graphics capabilities of UNIX in 2010 are so much
worse than, say, the computers I used in elementary school[1], in spite
of the great strides that graphics hardware has made.

NetBSD does need to delegate user-input events and control of the
display to processes in a systematic way, without the overhead and
complexity of X11.

How do we get there from here?  I envisage a global scene graph on
each display.  Corresponding to the root node and some other nodes
in the scene graph are "display capabilities."  One process grants a
display capability to another process by passing a file descriptor.  The
right conferred by a display capability is to add descendants to the
corresponding node in the scene graph.  A display server will add to its
scene graph nodes corresponding to a clipping shape, a rotation and/or
translation, a z-order, and an alpha level before passing the capability
corresponding to the new leaf node to a subordinate process.  The
clipping, rotation/translation, z-order, and alpha nodes all restrict
the subordinate process: it cannot draw out of bounds, and graphics
drawn by the display server within the same bounds (e.g., window
decorations) cannot be concealed.

Sounds to me from Wayland's descriptions that it may cover a lot of the
above already.

When you have such a simple and flexible model as scene graphs[2], you
can do use it as the drawing backend for X11, but you can also create a
lightweight handset UI, or a zoomable document-centered interface that
sidesteps WIMP conventions.

Dave

[1] I was in 3rd grade in 1986.  In 3rd grade, I remember that we used
    LOGO on Apple II computers.

    Just in case you think that I want to re-live my elementary
    school days on a NetBSD console: not really, no.  I think that
    on UNIX in 2010, the moral equivalent of LOGO would be for a
    UNIXy expression such as 'du -ks * | piechart' to draw the
    chart that one expects, without X11 or a window manager
    necessarily being in the loop.

[2] The implementation of such a model won't be easy.  Hardware
    graphics capabilities vary over a wide range, so a display
    capability that a software library provides by drawing
    pixel-by-pixel on one machine may be provided on a second machine by
    a device driver for a graphics coprocessor whose native interface
    deals in scene graphs.

-- 
David Young             OJC Technologies
dyoung%ojctech.com@localhost      Urbana, IL * (217) 278-3933