DS9

History of DS9 development

The first versions of DS9 were made available in 1999.

DS9 development was initially funded by a NASA Applied Information System Research Program grant: NAG5-3996, ``Future Directions for Astronomical Image Display'' with Eric Mandel as the Principal Investigator . The scope of work for this grant included, among other things, a set of portable and reusable widgets (frames, panner, magnifier, etc) that could be used to create custom astronomical imaging displays. DS9 then was the original product demonstration/reference implementation for this set of widgets [Joye & Mandel, 1999ASPC..172..429J] . However, DS9 was quickly adopted by the community and major observatories like NASA's Chandra X-ray Observatory and ESA's XMM Newton mission as the imaging software of choice. In the Final grant report, Mandel makes the following prophetic statement:

With support for its further development being provided by SAO, we are confident that DS9 will soon become the de facto image display standard in the astronomical community.

Over the next decade or so, DS9 development continued with Bill Joye as the developer and funding coming from several sources including SAO's High Energy Department funds and NASA/GSFC HEASARC grants. Funding for 3D rendering was provided by the JWST Mission Office at the Space Telescope Science Institute [Beck, Joye, and Conti, 2012AAS...22013517B]. In 2015, the Chandra X-ray Center at SAO took over full funding of the DS9 project which had become a critical component in multiple Chandra subsystems: standard data processing Verification and Validation, production of the Chandra source catalog, observation planning, and end-user data analysis. Chandra continues to fund DS9 development; however, with major anticipated reductions in the Chandra budget, support for DS9 is expected to be terminated.

DS9 typically has 1 release per year in late summer/early fall, and there are typically two or three beta releases throughout the year. As the only developer, Joye handles all aspects of the project from coding bug fixes and enhancements, testing, providing user support, generating documentation and project status reports. A small group of scientists and engineers provides input on project priorities and assists with some testing. The current WBS for DS9 looks like the figure below.

Work on DS9 has been presented at the Astronomical Data Analysis Software & Systems (ADASS) conference series. A collection of papers is available in NASA's ADS:

https://ui.adsabs.harvard.edu/public-libraries/47Zk2IAvRoOI7VFXs4R3RQ

Code evolution

Some examples of past significant changes to the code that addressed evolving community needs include:

• FITS format. The interesting thing about `standards' is that they change. While the original FITS standard was published in 1981, it has continued to evolve with the most recent major version, 4.0, being adopted in 2016/2018. Changes to the format include new data-types such as the 8-byte long, and new extension types such as compressed image formats.

• World Coordinate Systems (WCS). Not only has the format of FITS files changed, but the community established WCS conventions (meta-data) have also changed. New libraries needed to be integrated into the code to support the latest conventions.

• New detectors/missions means new requirements. The development of new hardware often precipitates a need for software changes. Trivial examples include needing to increase the numerical precision displayed to accommodate new milliarcsecond resolution instruments. More complex examples include support for missions such as JWST whose WCS cannot be expressed within the existing standards and requires non-standard processing. DS9 currently relies on the approximate WCS provided by JWST.

• Evolution of the Internet. From secure HTTP protocol to the demise of FTP these changes directly affect how DS9 interacts with online resources and how users obtain the software.

• Interprocess communication. A key element of the DS9 design was the ability for the application to communicate with other applications. Originally this has been through the "XPA" protocol which was designed and implemented under the same NASA grant that DS9 was developed. The astronomical community has now largely migrated to the "SAMP" protocol and much of the work on DS9 over the past few years has been directed towards this effort.

• User Hardware. Over the history of its development, DS9 has supported many different, some now defunct, operating systems: SunOS/Solaris, SGI/IRIX, IBM/AIX, HP/UX, DEC/Alpha, Linux, Windows, and Apple . It is also worth noting that it also has evolved from the days of 8-bit displays (limited to 256 colors and having to manage color tables) to today's 24/32-bit RGB/A displays.

Overview of Tcl/Tk

Tcl is a high level interpreted dynamic programming language; it is coupled with Tk, a graphical widget toolkit. Both packages are open source under a BSD-like licence and are written in C/C++ and in Tcl itself. Tcl/Tk was designed by John Ousterhout at Berkeley with a first full release in 1991. Tcl/Tk is currently developed by a distributed core team. SAO does not develop Tcl/Tk itself; we ingest the public release and distribute it with DS9 with only minor modifications.

DS9 code layout

DS9 is a Tcl/Tk application, with the GUI defined via Tcl/Tk code, but all of the real work is performed in C++. There are a number of parsers defined, written in YACC and LEX, for processing external data files such as regions files and colormaps, and all documentation is written in HTML.

All DS9 functionality is available via the GUI, or from the command line, SAMP, or XPA. Command line options, SAMP and XPA all support the same command syntax. SAMP is a xmlrpc based communication protocol, as defined by the International Virtual Observatory Alliance while XPA is an SAO-developed socket based communication protocol. Both are used extensively within the astronomical community.

The DS9 executable runs the main Tcl/Tk event loop which renders the DS9 user interface. Below is an example of the DS9 user interface with just some of the display elements labeled with their corresponding source code locations:

[Fig3]The Tcl/Tk DS9/library module provides the top level window in which all the other elements are displayed. In this example the "Advanced" view is being used. This module also handles all the events such as clicking to create new regions or panning the frame. The image itself is a custom frame widget written in C/C++. Similarly the magnifier, panner, and colorbar are all custom widgets written in C/C++. The AST library is used to compute the celestial coordinates used in the information panel. The overall look and feel of the user interface (font color, size, style, spacing, etc) is provided by the base "ttk" widget classes; the scidthemes provides a collection of ttk styles.

DS9's zipfs internal file system

On linux, the DS9 program is downloaded from ds9.si.edu as a single file. This file actually contains two concatenated sections: the DS9 executable proper, and a compressed file system containing the scripts and libraries. When DS9 starts up, it mounts this compressed `zipfs' file system in memory and accesses it.

The advantage of this approach is robustness for the user. The installation process is very simple: download a single file and put it somewhere in your path. It avoids complexities with unpacking distributions and with defining paths to libraries and shared files that might get moved somewhere the executable can't find them.

Paths in the internal file system are referred to in the code with a path of the form `zipfs:/mntpt/foo'.