/ Hikaru Date / Professor
/ Jintae Lee / Assistant Professor
The focus of the multi-media research in our laboratory is not the same as that of industry, which is actively developing various multi-media devices for the public market. Rather, our role is to initiate research toward the development of multi-media systems which can then be developed further at applied research laboratories and passed on to private corporations for marketing in a competitive environment.
The multi-media devices presently available add sound, still pictures, and moving pictures to traditional computer displays which show only text and diagrams. The addition of these few functions, however, offers remarkable convenience to the user of such multi-media devices. For this reason, it is widely believed that the impact of these devices on human society in the near future will be extremely great. However, ideal multi-media devices should go further than the crude ones at this initial stage. Future devices should have far more natural and simple human interfaces. Furthermore, they should have various convenient properties which strengthen human intellectual activity with the aid of subsidiary intellectual support.
The properties desirable for multi-media devices can be visualized on a graph with two axes. The horizontal axis represents a device's capacity for depth and variety of communication, and the vertical axis represents the capacity for naturalness and high fidelity. Ideally, multi-media devices should be far along both axes if they are to offer the most comfortable and efficient human interfaces. Unfortunately, progress along these dimensions has not gone far enough. Prolonged use of computers still fatigues the body and fails to match the depth and quality of communication that can exist between humans in face-to-face encounters.
For example, human beings can communicate with great ease and variety through the spoken word. Computers can't. In spite of the fact that speech synthesis by computers is almost perfect, speech recognition has not reached the level of practical application that natural speech can achieve. Furthermore speech recognition algorithms are overly sensitive to existing noise. The area of speech recognition is still in the primitive stages and needs to evolve further to make multi-media devices more effective.
Sign language is another means of human communication that computers cannot participate in as easily as those who are forced to use it because of hearing impairment. If a computer could be made to communicate via this medium, far more people could access intellectual services and actively participate in multi-media that have been unable to do so thus far. Pioneer research to incorporate this dimension of communication as far as synthesis and recognition technology started in our laboratory.
As for the dimension of high fidelity, the digitalization of high definition TV is one example of progress that has been made along the vertical axis. Another area that shows potential is research in virtual reality which can capture suspected experience of the three dimensional world. Our lab is contributing to the research in this area with studies of high fidelity technology related to visual and auditory sensations which allow natural, integrated spatial sensation with minimal mental effort.
Signal processing technology is indispensable for the generation, processing, and recognition of visual, auditory and control signals. Furthermore, multi-media devices are preferable to work in real-time. These requirements can be fulfilled not only by high computing processor speeds and large memories, but also by fast algorithms. Therefore, the study of signal processing is fundamental in our laboratory.
This year, in order to develop a practical speech recognition machine, Professor Date, in cooperation with Professor Sugiyama of the Human Interface Laboratory, developed digital hardware for a noise canceling microphone based on a new principle. Another member of our laboratory, Professor Lee continued his sign language research and presented a paper at an international conferences in Europe.
Refereed Journal Papers
This paper describes a new method which employs a hand model and static stereo video images to automatically analyze hand posture. A model-based framework is proposed in which the movement of the hand model is guided by internal constraints and external driving forces derived from the video images. The internal constraints were formalized based on anatomical data of the human hand and then incorporated into the model. To transform the hand model to an image having the posture of a real hand, characteristic points on the hand are identified from the images, and virtual springs are implied which pull the characteristic points on the hand model to goal positions. Experimental results are presented showing hand shapes of letters used in American Sign Language (ASL).