Rate-Distortion Theory and Deterministic Annealing

Signal Compression Laboratory Research Project

 

Researcher: Ertem Tuncel
Faculty: Prof. Kenneth Rose
Research Focus: The deterministic annealing (DA) algorithm can be utilized both for VQ design and for rate-distortion R(D) curve computation for memoryless sources and continuous alphabets. The only difference between the two tasks is that, in VQ design, phase transitions (PTs) are not allowed after the desired number of reproduction vectors is reached, whereas in R(D) curve computation, all the PTs are allowed.

During the design, two types of PTs are encountered. The first type is where a reproduction vector is split into two or more reproduction vectors, and in the second type, reproduction vectors need to grow out of zero mass. Splitting-type PTs can always be followed; even analytical expressions can be derived for whether it is time for a splitting-type PT or not. This is not the case for a growing-type PT, and hence, DA algorithm does not guarantee optimality. For R(D) curve computation, this may not be important because a miss of growing-type PT may be compensated by a splitting-type PT afterwards. However, for VQ design, examples exist where missing a growing-type PT results in a suboptimal codebook. Another problem for VQ design is, because of prohibiting PTs after the desired number of codevectors is reached, even if all the PTs before that temperature are caught, the algorithm may still end up with a suboptimal codebook.

In this work, some related open problems are being attacked:

1) A DA algorithm for R(D) curve computation that guarantees optimal solution at every temperature is not known. This can only be done by developing a way to catch the growing-type PTs. Some heuristic algorithm for this purpose is developed.

2) A DA algorithm for VQ design that guarantees an optimal codebook is also not known. One has to understand the effect of limiting the size of the reproduction set in R(D) computation in order to solve this problem.

3) It is not yet proven that, as the temperature decreases, the number of the reproduction vectors is monotonically non-increasing. The solution of this problem may help us to solve the problem of "successive refinement of information" introduced by Equitz and Cover.