Preliminary Specifications for a High Performance Byte Raster Display Terminal Hans Moravec Stanford University June 12, 1977 Introduction: This document is a preliminary set of specifications for a high performance graphics terminal, designed to produce high resolution grey scale and color, as well as sound, on NTSC standard television displays. The basic terminal is externally a box with a keyboard and some connectors and a few lights and switches. The connectors are for power, modem or phone line, video out, audio out, rf out (audio and video modulated onto a tv carrier), I/O bus (or unibus). It should be as light and compact as possible. Optional accessories would include a high quality outboard tv monitor, audio amplifier, mass storage device, printer, etc. The terminal contains a processor, assumed to be a PDP-11, which drives the display, handles communications via a modem and possibly drives other outboard input, output and storage devices. 256 K bytes of main memory, accessible through a memory map (this is the same amount as the byte raster memory) would not be excessive. This memory will be required to store font definitions for text display, and phonemes and other sounds for audio output, as well as medium large display programs. The display is built around a 512 by 512 byte raster memory, each byte specifying the intensity of a spot on the screen. The sample rate is three times the color subcarrier frequency, so it is possible to synthesize the color subcarrier, and thus produce an NTSC standard color image. Alternately the high sample rate can be use to produce a superior resolution (about 5.5 MHz) black and white image. This memory should be mappable into the address space of the processor. The audio is generated by an I/O device which is commanded to feed a given number of words from the 11's memory to an audio rate D/A at one of a number of fixed rates ranging from about 1 to 50 KHz. If desired it can be made to generate an interrupt when it finishes the requested number of samples. A bootstrap loader, which can be made to load from an audio cassette playing into the modem or from a suitable device plugged into the I/O bus, is contained in a ROM in the PDP-11, and can be activated with a switch on the keyboard. Why: With these specifications, and suitable software, the terminal can exceed the performance of most existing text and graphics terminals, except for very quickly moving images, for which vector displays are better. It also exceeds the growing capabilities of home computer systems, some of which have limited graphics (usually by means of a programmable character generator). It will also exceed the abilities of TV games, which have always had limited graphics and sound ability, and are now evolving programmability and keyboards. Most terminals are capable of generating a bell or a beep to alert the user to important messages. With the more general sound generating ability suggested here, a variety of sounds, for different events can be generated, as is feebly demonstrated by some TV games. Speech generation is possible with a reasonably small algorithm and a stored set of phonemes. Limited music can also be produced. The grey scale display can be used to reproduce arbitrary pictures, but is also (surprisingly) a great asset in simply displaying text. About four times as many readable characters can be put on a grey scale raster than on a bit raster with the same number of samples. This is because the intensity can be used to simulate higher resolution. Samples fractionally covered by a hypothetical high resolution character are fractionally shaded. The slightly blurry rendition this produces appears to be effortlessly reconstructed by the human visual system much in the same manner the classical portrait of Lincoln reduced to a 5 by 5 array of grey squares is recognizable at a distance. Pushing things to extremes, the 512x512 image can display about 80 lines of 160 characters each. The grey scale also allows apparently higher resolution for line drawings, with line that gradually fade from one column to the next along their length. The most exciting possibilities involve generating varyingly shaded areas, as in realistic pictures of 3D objects. The device can clearly be used as a stand alone personal computer, but it will be even more useful as a remote (or not so remote) terminal. The idea is that a graphics system be distributed between the terminal and the central computer so as to minimize the amount of communication needed. Subroutines in the terminal would be activated by commands and data from the central machine. There should be a protocal for loading new routines from the communications link into the terminal, and a bulletproof procedure for alerting the central computer in the event of the terminal crashing. This is mostly a matter of software. Byte Descriptions: Each sample in the 512x512 raster is an eight bit byte. The bytes are used to drive an eight bit D/A, after first being transformed by a 256 (2^8) word by eight bit memory, called the intensity map. That is, an eight bit sample coming from the display is used as an address into the 256 word intensity map, and the contents of that address are used to drive the D/A. This allows programmable intensity and brightness adjustments, as well less linear operations such as selecting single bits from the samples, so that the display can be used as 8 single bit rasters. Diagram: The overall organization of the display is as follows: +-------------------------------------------+ | +-----------------+ | +--+---+-+ +-+----------+ | | | +------+ | | +----+------+ | | | | | | | | | | | dis- | | 512x512 | | 256x8 | | PDP-11 | | play | | x8 | | | +------+ +--+ +---+ +-----+ intensity | | | | |cont- | | raster | | map | | | | |roller| | memory | | | | | | +------+ | | +-----+-----+ | +------+-+ +------------+ | | | | | +---+---+ +---+---+ | | 16 bit| | 8 bit | | | audio | | video | | \ rate / \ rate / | \ D/A / \ D/A / | \ / \ / | +-----+ \ / \ / | |modem| +------------------+ +-------------------+ +--+ | | +----+-+----+ | | +--+--+ | | RF | | | | | | modulator | | | | | +-----+-----+ | bus phone audio | video i/o i/o out RF out out Processor access: The 512 by 512 raster, and the rest of the processor's memory, can be accessed through a small special purpose processor which allows rectangular subareas of rectangles in memory to be dealt with conveniently. The processor has two identical ports, allowing two rectangles of different sizes in different parts of memory to be accessed simultaneously. Each port of the processor is controlled by 14 unibus locations. The suffixes 1 and 2 indicate which port: HA1, HA2 high order bits of address of top left byte of outer rectangle LA1, LA2 low order 16 bits of outer rectangle address OH1, OH2 height, in bytes, of outer rectangle OW1, OW2 width of outer rectangle RI1, RI2 position within outer rectangle of top edge subrectangle RJ1, RJ2 position of left edge of subrectangle being accessed RH1, RH2 subrectangle height RW1, RW2 subrectangle width BI1, BI2 row of bytes within subrectangle BJ1, BJ2 column of bytes within subrectangle BD1, BD2 byte data in/out BIINC1, BIINC2 each reference to BD increments BI by BIINC BJINC1, BJINC2 how much to increment BJ for each reference to BD BIOVR1, BIOVR2 how much to increment BI when BJ overflows BJOVR1, BJOVR2 how much to increment BJ when BI overflows BJ 0 1 2 3 RW-2 RW-1 +----+----+----+----+--- ----+----+----+ | | | | | | | | 0 | | | | | | | | +----+----+----+----+-- . . . ---+----+----+ | | | | | | | | 1 | | | | | | | | +----+----+----+-- --+----+----+ | | | | | | | | | | | | BI . . . . . . . . . | | | | | | +----+----+---- --+----+----+ | | | | | | | RH-2 | | | | | | | | +----+----+----+----+-- . . . ---+----+----+ | | | | | | | | RH-1 | | | | | | | | +----+----+----+----+--- ----+----+----+ Summarizing, RI=0 refers to the top row of the outer rectangle, RJ=0 is the leftmost column. BI=0 is top row of bytes within the subrectangle, BJ=0 is leftmost column. Data written into or read from BD shows up in or comes from byte [RI+BI,RJ+BJ] of the outer rectangle, which is byte HAx2^16 + LA + OWx(RI+BI) + RJ + BJ of the total memory. BI and BJ are incremented after the read or write (or read-modify-write) by the following algorithm. BI := BI+BIINC BJ := BJ+BJINC T := BJ if BI>RH-1 then BJ := BJ+BJOVR if T>RW-1 then BI := BI+BIOVR if BI<0 then BJ := BJ-BJOVR if T<0 then BI := BI-BIOVR if BI>RH-1 then BI := BI-RH if BI<0 then BI := BI+RH if BJ>RW-1 then BJ := BJ-RW if BJ<0 then BJ := BJ+RW The multiplication by OW required in the address calculation could be eliminated if RH, RI, BI, BIINC and BIOVR were expressed in multiples of OW. This has the problem that the data in these registers could exceed 16 bits for large rectangles. This arrangement allows subrectangles within larger rectangles to be filled by row [BIINC=0, BJINC=1, BIOVR=1, BJOVR=0], by column [BIINC=1, BJINC=0, BIOVR=0, BJOVR=1], by diagonal [BIINC=1, BJINC=1, BIOVR=0, BJOVR=1], forward, backward (by making various parameters negative), and in less conventional patterns. To access the raster memory OH and OW are both set to 512 and HA,LA are set to wherever in the extended address space in the memory the display memory is wired. Implementation: The raster memory contains 512x512x8 bits. This would require 128 16K RAMs. If these were read out 8 bits in parallel, the cycle time would need to be 90 ns. Reading out 16 bytes at once, into a higher speed shift register, increases the time available to 1.4 microseconds, within the specs of most existing dynamic MOS RAMS. 16 parallel bytes implies that all 128 RAMs are involved in each cycle, and that the buffer register is organized as 8 parallel loadable shift registers, each 16 bits long. If the cycle time of the memories were below 700 ns, the buffer memory requirements could be halved. The access rate of the display is such that no explicit refresh circuitry is required. The longest interval between refreshes, about half a millisecond, happens during the vertical retrace. The 256 word intensity map memory is accessed at random, each sample time, every 90 ns. It is essentially impossible to increase this number, so a fast memory is needed. Bipolar RAMs with cycle times as low as 60 ns are available, in capacities up to 1K. Raster organization: The sample rate is 3x3.579545 MHz, three times the NTSC color subcarrier frequency. The sample period is thus 93.12 nanoseconds. The total raster, including all sync and blanking intervals, is 525 scanlines high, with 682 samples per scanline. Each scanline consists of 100 sample times of horizontal sync pulse and back porch, including color burst, followed by 28 blank (black) samples, then 512 data samples read from memory, another 28 black samples and 14 samples devoted to sync pulse front porch. +--+ | | +- | +-++--...................................................--+ | | 100 |28| 512 |28|14| sync blk data blk sync &color 512 of the scanlines actually carry data. The vertical sync interval occupies about 7 scanline times in each 262.5 scanline interlace field. There are thus 512x512 = 256,000 bytes of data displayed. On most tv monitors only 490 or so of the scanlines are actually visible. There is a unibus accessible control bit, called COLOR, which if on causes a ten cycle color burst be be inserted at the appropriate time in this raster. The phase of this signal should probably be the same for each scanline within a retrace field, and 90 degrees out of phase between the even and odd fields. The standard broadcast technique of alternating the phase each frame is not possible with this organization, because it requires the color carrier in the data portion of the picture to change each frame. Even more: These specifications, ambitious as they may sound, represent a considerable amount of self restraint. Although they are difficult and expensive to meet now, in about 5 years (when there are megabit RAM chips) I expect sub $1000. TV games to meet many of them. Things that are desirable but possibly currently unreasonable: 1: More processing power. Many (most) of the most exciting types of graphics, involving things like shading of 3D objects and motion, are very slow, even on what are currently considered big computers. There is probably no upper limit on the amount of computational power that could be used effectively. 2: More storage. Increasing the bits per sample in the raster memory from 8 to 16, and the size of the intensity map memory from 256x8 to 65K by 16 and adding another video D/A or two would allow color images with as much resolution as the proposed B/W capability. This would require interfacing a color monitor at the individual color amplifiers. It also allows tricks such as storing four 4 bit B/W pictures and switching between them rapidly by changing the intensity map. Increasing the processor's main storage would make easier convenient picture languages dealing with arrays of intensity values. I have in mind a particular system that is greatly limited if the main memory is only as big as the raster memory. 3: Larger address space. The choice of a PDP-11 as the main processor is based on expediency, not because it has the perfect organization. The complicated nature of the raster memory access processor is partly due to the fact that 16 bits is inadequate to address the 262 Kbyte raster. The same limitation will complicate the implementation of good graphics software for the terminal. It seems to me that a 32 bit address would not be excessive for this application, allowing convenient implementation of a large virtual memory on mass storage, and room for expansion as memory costs decline. 4: Video and audio input, as well as output. If sync detection circuitry and a fast A/D were added, the raster memory could be used to digitize an external tv signal. The image could then be examined and perhaps transmitted by a program in the terminal. Similar considerations for audio.