/ Makoto Kobayashi / Professor
Since the Performance Evaluation (Research) Laboratory (PERL) was opened in July 1993 and started from scratch, most of our effort in the 1993 was devoted to get PERL into shape in terms of equipment, research projects, and people.
Performance of a computer is determined not only by the hardware but also by the workload. That is, performance is determined by the interaction between hardware and software. Today's high-performance workstations, let alone main frame computers, are very complex in design to get more performance. They commonly employ such techniques as single or multiple CPU pipelines, branch target buffer, instruction prefetching, and single- or two-level cache memories. These techniques take advantage of particular aspects of program behavior. For example, cache assumes locality of reference which states that a program tends to concentrate its memory references to a small subset of its data space during a relatively short period of time. Pipeline interlocks depend on how different instructions are executed sequentially. As such, performance of a computer highly depends on its workload. Workload characterization is essential to study performance of today's computers.
Analytical models, such as the queueing network model and the petri net model, are not detailed enough to study performance at this instruction level. We need to use the cycle by cycle, register transfer level (RTL) simulation model, which is commonly driven by the instruction traces or memory reference trace. Hence, we needed to select workstations which are capable of running large simulation programs efficiently. A set of benchmark tests were conducted on the IBM RS/6000 model 580 and the HP Apollo 9000 series 700 model 735. The benchmark job was a large simulation program written in SES/Workbench. Based on this benchmark test, the IBM RS/6000 model 590 was chosen. An internal report was written on the benchmark test.
Two major research projects were initiated jointly with other laboratories: (1) performance evaluation of multiprocessor systems and (2), workload characterization of the database systems.
A major difficulty in evaluating performance of a multiprocessor system is modeling the workload which drives the hardware model. At the RTL level, the workload is represented by the instruction traces. Unlike in a uni-processor environment, we cannot use the same instruction trace to evaluate different architectures in a multiprocessor environment because the timing differences in, say, spin locks and even more seriously low level workload, e.g., process execution, are different. In order to overcome this problem we proposed a generative control flow graph model which can generate different low-level workload in different multiprocessor environments. This project has been jointly investigated by Professors Makoto Kobayashi, Shietung Pen and Sergey V. Ten. Professors Peng and Ten belong to the Distributed Parallel Processing Laboratory.
The goal of the second project is to characterize the database systems, initially in a uni-processor environment and eventually in a distributed, parallel environment. Of particular interest is the instruction and data memory reference behavior. First, instruction traces of an existing database system are generated. Then, the memory reference behavior is studied and a new memory reference model is proposed. By using both original instruction traces and also the newly proposed memory reference behavior model, the memory subsystem is evaluated for the hardware bottlenecks. This project has been jointly investigated with professor Subhash Bhalla of the Database Systems Laboratory and professor Omar Hammami of the Computer Architecture Laboratory. A research collaboration agreement was obtained from Oracle Japan.
As a part of the effort to increase the awareness in performance evaluation, talks were given at a system evaluation seminar, at the University of Hiroshima undergraduate division, and at the information center at the University of Hiroshima.
A weekly PERL meeting has been held every Thursday afternoon to discuss technical issues of any performance-related subject.
Academic Activities
Makoto Kobayashi, NEC, August 1993.
Gave a talk on performance evaluation of computer systems at the NEC central research laboratories.
Makoto Kobayashi, Hitachi, October 1993.
Gave a talk on performance evaluation of computer systems at the Hitachi Systems Laboratory.
Makoto Kobayashi, Hiroshima University, February 1994.
Gave a talk on software-hardware interaction at the engineering department and also at the information processing center of Hiroshima University.
Makoto Kobayashi, Information Processing Society of Japan, March 1994.
Gave a talk at a Systems Evaluation meeting.