From a historical perspective, positioning technology innovation has always played a key role in the industrial technology revolution. Recently, with the rapid advancement of Industry 4.0, which completely alters the traditional thoughts and operation modes of manufacturing, the need for advanced and low-cost localization schemes has become even more critical. Furthermore, for intelligent living and assistive purposes, technologies for supporting the daily life of aged people and for realizing dynamic and green architectures also relies on advanced indoor localization of moving agents. This article briefly introduces the development of indoor localization techniques based on Nintendo Wiimote, which was originally designed as a game machine. Starting from the efforts conducted by the NCKU research teams in intelligent architecture for achieving structure adaptivity, human-building interaction, and energy-saving purposes, such a localization technique provides essential coordination between different building elements. Subsequently, special 2D and 3D Wiimote localization schemes were developed for navigation, control, and task planning of indoor mobile vehicle groups after overcoming major limitations. With those schemes ready, it was then possible to track the attitude and location of key manufacturing equipment in factories to fulfill essential manufacturing or logistic task planning. The application was thus further extended to localize the 3D objectives and simple computer animation creations for subsequent cyber-physical integrations, which is important in current Industry 4.0 applications in manufacturing, logistics, and virtual reality, as well as digital twins. It is expected that the developed technology could play key roles in promoting intelligent manufacturing and smart living in the near future.

From a historical perspective, positioning technology innovation always plays the key role in industrial technology revolution. For example, the global positioning system (GPS) promotes the high-end navigation business of vehicles and drones. Recently, with the rapid advancement in Industry 4.0, which completely alters the traditional thoughts and operation modes in manufacturing, the need for advanced and low-cost localization schemes has become even more critical for localizing and coordinating key factory equipment such as AGVs and robot arms, to conduct necessary manufacturing task planning. Furthermore, for intelligent living and assistive purposes, technologies for supporting the daily life of aged people and for realizing dynamic and green architectures also rely on advanced indoor localization of moving agents for realizing navigation, motion control, collision avoidance, and trajectory planning tasks (Figure 1). 

This article mainly provides a brief introduction to indoor localization using a game machine (Figure 2): the Nintendo Wiimote, which was originally developed as a man-machine interaction medium for computer games (Figure 3). On the board, an IR sensitive CMOS-camera is responsible for detecting the IR light source and its position. It is therefore possible to extend the original game machine nature to localize indoor moving objects. In fact, a group of researchers at Carnegie-Mellon University (CMU) has identified the potential of Wiimote in other applications such as electric whiteboards and multi-media applications, and designed a case study for demonstration. Since then, various applications have been proposed. In 2010, NCKU researchers, under the support of MOST and led by Prof. M. C. Tsai of the ME department, proposed a new concept of dynamic architecture technology, where traditional architecture is integrated with sensors, actuators, and communications, for achieving structure adaptivity, human-building interaction, and energy-saving purposes. Simply speaking, the proposed building could change its configuration according to environment excitation and its needs. For achieving the design goals, numerous building elements have to be coordinated. As a member of the research team, the Wiimote localization technique was then proposed to assist with development by accurate positioning and subsequent task planning. The concept earned various prizes and honors. To the best of our knowledge, this was also the first Wiimote application beyond the original game and multi-media targets. With the successful demonstration of its localization capability, more challenging indoor localization tasks were conducted for intelligent manufacturing. These achievements were chosen to be presented during the press conference a few years ago that was organized by the Ministry of Science and Technology.

Figure 1. Intelligent architecture and dynamic structural units

Since those building elements could be treated essentially as group mobile robots, it is natural to extend the developed techniques to robot motion control. Our group thus investigated the possibility of using Wiimote for the indoor localization of group mobile robots by developing the associated navigation, obstacle avoidance, tracking and following, and group manipulation, as well as trajectory feedback control of mobile vehicles. The core issue is the realization of a multi-zone 2D indoor localization scheme to extend the arena up to the size of an entire indoor living space. Furthermore, a novel man-machine interaction module was also designed to guide robot motion remotely via Wiimote by simply recognizing the number of IR LED clusters and using it as the basis in task planning.

Figure 2. 2D Wiimote localization for mobile robot indoor task planning applications.

Figure 3. Man-machine interaction using Wiimote

Subsequently, Wiimote 3D localization based on stereo vision was also proposed by the team, and the localization performance was investigated. With the technique ready, it is possible to estimate the orientation of robot arms in real time without using an expensive camera system and image processing (Figure 4). Furthermore, another major obstacle: the maximum allowed observed IR LED clusters limited by the available Bluetooth channels, was also overcome by the team. The application was then further extended to localize the 3D objectives such as robot arms carried by AGV and simple computer animation creations for subsequent cyber-physical integrations, which is important in current Industry 4.0 applications in manufacturing, logistics, and virtual reality, as well as digital twins. 

Figure 4. 3D indoor localization and collaboration between robots

As addressed above, in modern intelligent manufacture planning, accurate attitude and position information is one key concern for subsequent equipment coordination control, task planning, and establishing a safe man-machine working environment. The developed Wiimote technology could play key roles in promoting intelligent manufacturing and smart living. Our team will continue to engage in this topic to contribute to society and to explore new applications. It is worth stating again that technical advancement begins from the breakthrough of fundamental techniques.