| 1998 Q1 | This is a status report for the Parallel Software-Only Video Effects Processor Project which is part of the Millennium Project. We have completed a high-level design of the software architecture for the project. The design decomposes the problem into three semi-independent parts. We are now starting to formalize the interface between the different components. In addition, we have implemented mechanisms for exploiting temporal parallelism for video effects processing. Intel donated equipment was used to develop and test these mechanisms. Using the Berkeley NOW we have measured the performance of these mechanisms. The design and initial performance measurements have been reported in a paper submitted to the ACM Multimedia 1998 Conference. A copy of the submitted paper can be found at http://www.cs.berkeley.edu/~kpatel/mm98-c-144.ps. |
| 1998 Q2 | We
are developing software to improve the quality of live video broadcasts
and to support postproduction for multimedia authoring. Image and video
processing are good examples of computationally intensive multimedia operations.
A parallel, softare-only video-effects system is being developed that will
run on a Network-of-Workstations (NOW) composed of general-purpose processors.
The system will be extensible and scalable. Functional, temporal, and spatial
decomposition, dynamic workload balancing, compressed domain processing,
and use of special-purspose instructions (e.g., MMX) will be exploited to
achieve real-time high-quality results [1].
During the past year we developed the infrastructure to support functional, temporal, and spatial parallism. We investigated the problem of how best to decompose video streams, called the "selector function," transmit the substreams to effects processors, and recombine the substreams when using temporal parallelism, called the "interleaver function." We investigated the use of a centralized versus a fully-distributed algorithm for the selector function and concluded that the centralized algorithm worked well enough (see below). Consequently, all frames are multicast to all effects processors (EP) and each EP selects the frames on which it will operate. We developed an adaptive recombination algorithm that works for dynamically varying streams (e.g., frame rate, interframe latency, out of order delivery, etc.). We implemented a cross-dissolve effect to measure the performance and latency of the centralized and distributed selector functions and to measure the buffer latency and frame drop percentage of the interleaver function. Using a four-way EP decomposition, we measured that 8% of the frames are lost when latency is bounded to one frame time. A paper describing this work was accepted for publication at the ACM Multimedia '98 Conference [2]. We also implemented and tested spatial parallelism. The selector function for spatial decomposition cannot be centralized because the output of an effect computed on a subregion of the frame may depend on all pixels in a frame (e.g., affine transformations composed of scale, rotate, and translation operations). Moreover, a compressed representation of the subregion may refer to pixels outside the subregion (e.g., difference coding of DC terms in MJPEG compression or motion vectors in MPEG compression). Consequently, we send complete frames to each EP and let the processor access the subregion required. We also developed a "semicompressed video" (SC) RTP payload format that simplifies the recombination of the subregions produced by the EP's. A paper has been submitted for publication that describes the SC payload format, compares the size of an SC encoding of a frame to the MJPEG encoding, and compares resources required to perform the cross dissolve effect used in the previous temporal decomposition paper [3]. During the coming year we expect to deploy the effects system in the "Berkeley Multimedia, Interfaces, and Graphics Seminar" and to continue the development of the system. Specific tasks are to develop cost functions for different parallel processing primitives so that an algorithm can be developed to automatically produce a parallel representation for an effect that will meet the resource requirements (e.g., maximum latency, throughput, and number of EP's). Another problem to be addressed is the development of a general control mechanism that will allow a director to control an effect. This control mechanism will be integrated with a director console application we are developing as part of the Internet MBone Broadcasting System being developed for the Berkeley campus [4]. Web page for more information http://www.bmrc.berkeley.edu/projects/ibs. REFERENCES [1] Tina H. Wong, Ketan Mayer-Patel, David Simpson and Lawrence A. Rowe, "A Software-Only Video Production Switcher for the Internet MBone" SPIE Multimedia Computing and Networking, January 1998. [2] Ketan Mayer-Patel and Lawrence A. Rowe, "Exploiting Temporal Parallelism For Software-only Video Effects Processing," to appear Proc. of The Sixth Annual ACM Intl. Multimedia Conf., September 1998. |