We show the demonstration video of our research in Youtube:  





 Currently, we are developing a mobile robot called "Goron". One of the functions of this "Goron" is to send the video image of the "Goron" camera wirelessly to the computer, and aslo send the result of gesture recognized by the computer to "Goron" in real time, and operate "Goron" a person tries to do. The gesture recognition is performed with the algorithm of "Time-space Continuous DP".
If we implement the recognition algorithm with FPGA etc., we can make gesture recognition of a robot "Goron" without sending wirelessly video data to an outside computer.

  "Time-space Continuous DP" was developed by us before, and here it is used to recognize gestures from a moving image of a single camera mounted on a mobile robot. At the present time, the gesture instruction is performed by recognition of the four actions instructing the mobile robot to turn clockwise, counterclockwise, and switch on and off the light mounted on the robot. Currently, the number of gestures to identify is small, then we would like to increase that number in the future.

 Normally, a mobile robot is performed by either or both of autonomously determining its movement, or deciding by human instruction. Automatic driving cars are the former. On the other hand, mobile robots involved with humans, such as nursing robots, often want to move according to human instructions. At that time, the human side usually has two instructions. It is voice and gesture. Although the voice is convenient, the current technology is the restriction that when a robot is separated from a human, a microphone close to the human side must be used. On the other hand, gesture has no such restriction, but in the situation where both robot and human are moving, technology to well recognize human gesture has not been established yet.

  Conventionally, a mobile robot equipped with a laser, an ultrasonic sensor, a kinetic sensor, etc. detects and keeps track of a stationary person in the vicinity. It can be said that there is rarely a research that a mobile robot equipped with a moving image of a single camera recognizes a gesture of a moving person in the surroundings and operates accordingly. The reason is that there is no technique for recognizing a specific gesture without specifying the start and end times, in a moving background and in an environment with an unspecified number of moving people.

Laboratories and Groups

Research Category

Areas of Activity
Robot Image Processing Intelligent System Man-Machine interface
Related Keywords