Visualization for HPC
What is Visualization?
Visualization is the process of converting large amounts of complex, multi-dimensional data into images so people can more quickly and easily see patterns and anomalies in the data. Visualization technologies are widely used within the HPC community to enable better understanding of the ever larger data sets that computer simulations and sensor networks are creating.
Visualization for HPC Goals
The OpenSolaris Visualization Project has three primary goals:
- Provide tools for building visualization server systems that can handle these massive amounts of data with sufficient performance. Many tools for scalable visualization (defined as harnessing the capability of multiple graphics devices to deal with very large data sets) are available in the open source community, primarily deployed on Linux and Windows platforms. One goal of the Visualization project on opensolaris.org is to provide a complete integrated and tuned stack of software for scalable visualization on Solaris.
- Provide easy access for any user anywhere to such systems, the programs these systems run, and the images they produce. These users can use many different types of client systems—Sun Ray™ thin clients, notebook and desktop PCs and Macs, and workstations. A second goal of the proposed visualization project is to provide software for enabling remote visualization and collaboration.
- Provide additional visualization applications and tools for building, analyzing, or tuning visualization applications on Solaris.
Techniques and Technologies
Server-Side Graphics
The primary techniques for building visualization servers are:
- Parallel Architectures: Using multiple graphics cards for one program, sharing the load. These multiple graphics accelerators could be attached to a single system (with massive memory and many CPUs to render large data). Or a cluster of computer nodes with graphics accelerators can form a rendering system that can work together on the problem.
- Parallel decomposition: A problem can be split among multiple graphics cards in the application's data space or in display space.
- Open Software Architecture addresses server-side rendering at middleware, API, or application levels. The following open source components are frequently used:
- Chromium flexible framework for scalable real-time rendering on clusters
- Paraview Parallel Visualization Application
- Open Scene Graph 3D graphics toolkit
- DMX (Distributed Multihead X).
The initial stack consists of Chromium, ParaView, OpenSceneGraph, DMX, and OpenMPI, with pre-compiled binaries, installation and configuration scripts, and documentation on how to put a large-scale visualization system together. Download Scalable Visualization (Solaris Packages) here.
Remote Visualization
The primary techniques to provide easy, high-performance access from remote networked clients are:
- Transmitting images, rather than graphical data, from servers to clients.
- Compression: By compressing images on the server and decompressing them on the client, more-plentiful CPU cycles substitute for limited network bandwidth. This generally increases individual performance and reduces network bandwidth consumption.
Open Source Software technologies used to address remote graphics and collaborative application use are
- VirtualGL, which interposes on unmodified Unix 3D applications, redirecting its 3D commands onto a server-side 3D graphics accelerator, reading back rendered images, and converting the application images into a video stream with which remote clients can interact to view and control the 3D application in real time. The video stream is normally compressed using server-specific optimization.
- TurboVNC (Virtual Network Computing with accelerated JPEG compression).
Visualization Resource Management
Sharing visualization resources among multiple users and uses requires resource management, to avoid oversubscribing a system or graphics accelerator, especially if there are multiple servers or graphics boards on those servers. Without management, how would a user know which resources to use? A case where one system is overloaded while another sits idle is easily imagined. Load balancing is needed.
Allocating an entire system to one user at a time is conveniently easy, but if the system has sufficient power for multiple simultaneous users, more sophisticated resource allocation is required.
OpenSolaris visualization has extended the Open Source Sun Grid Engine software to allocate graphics resources, as well as CPUs and memory. Sun Grid Engine can select a suitable, lightly-loaded server to run your application. The Visualization extensions can also select a lightly-loaded graphics accelerator on that server.
The initial software stack consists of VirtualGL, TurboVNC, and scripts that allow for the easy integration with Sun Grid Engine to manage graphics resources on a grid of visualization servers.
Remote Visualization Downloads
Pre-compiled binaries with installation and configuration scripts may be downloaded here (registration required). Documentation, including Release Notes describing what is new in release 1.1, is available here.
Related Sun™ Products
Sun Visualization Systems
Sun Scalable Visualization Software is bundled with hardware as the Sun Visualization System. Sun "Customer Ready Systems" or a partner will normally install, configure, and test the hardware and software.
This OpenSolaris Visualization Project allows others to use the Scalable Visualization software without purchase of bundled hardware. We welcome others to share their results of trying the software (or alternative component versions) on various platforms. Download Scalable Visualization (Solaris Packages) here.
Shared Visualization Software
Sun Shared Visualization Software is available for free download here (registration required) and use. Support plans are available for purchase.
A wide variety of clients are supported, including Solaris, Linux, Mac OS X Intel, Windows, and Sun Ray™ thin clients. Documentation, including Release Notes describing what is new in release 1.1, is available here.
Current Release
More information on the features and components of the current release, including what's new, is available here.
GLanalyzer Software
GLanalyzer is a tool for recording and analyzing OpenGL applications. Using GLanalyzer you can make a recording of the OpenGL API calls your application makes. The recording can then be used to reproduce a bug, to analyze performance problems or to make a source code version of the recording. Other uses include collecting primitive counts, fill rate and other statistics. GLanalyzer has a modular data flow design to allow traces to be studied in a variety of ways.
You can download the source for GLanalyzer here.
OpenSolaris Visualization Project Information
View the leaders for this project