TELEOPERATED SYSTEMS: A PERSPECTIVE ON TELESURGERY APPLICATIONS

SISTEMAS TELEOPERADOS: UNA PERSPECTIVA DESDE LAS APLICACIONES EN TELECIRUGÍA

SISTEMAS DE TELEOPERAÇÃO: UMA PERSPECTIVA SOBRE APLICAÇÕES TELECIRURGIA

Róbinson Jiménez Moreno^1,2, Fabio Andrés Espinosa Valcárcel¹, Darío Amaya Hurtado¹

1 Universidad Militar Nueva Granada, Bogotá, Colombia.
2 Dirección para correspondencia: robinson.jimenez@unimilitar.edu.co.

Recibido 27 de noviembre de 2012. Aprobado 8 de agosto de 2013.

ABSTRACT

Teleoperation had become one of the most prolific areas of research due to the great number of applications in which the remote operation paradigm can be applied. Teleoperation is made from several disciplines, areas or techniques, ranging from communications to systems control and from virtual reality to digital signal processing. Hence, individual advancements in these areas could be applied to developments in teleoperation systems. This article presents a compilation of important terms and concepts.

It also presents a compilation of the state of the art in teleoperation systems, with special emphasis in telesurgery applications and related research studies in the area comprising image processing and cooperative robotics, which have been used to increase the effectiveness in telemedicine systems and to increase the quality of surgical procedures.

KEY WORDS: Image processing; Telemedicine; Teleoperation; Telesurgery.

RESUMEN

La teleoperación se ha convertido en una de las áreas de investigación más prolíficas, debido al gran número de aplicaciones en las que se puede aplicar el paradigma de operación remota. La teleoperación está conformada de varias disciplinas, áreas o técnicas, desde comunicaciones hasta control de sistemas y desde realidad virtual hasta procesamiento de señales digitales.

Así, avances individuales en estas áreas podrían ser aplicados a la evolución de los sistemas de teleoperación. En este artículo se presenta una compilación de términos y conceptos importantes en el área. También se presenta una recopilación de los más recientes avances en sistemas de teleoperación, haciendo un especial énfasis en la telecirugía y en las investigaciones en esta área que durante los últimos años, hacen uso del procesamiento de imágenes y de la robótica cooperativa para aumentar la efectividad de los sistemas de telemedicina con el fin de incrementar la calidad de los procedimientos quirúrgicos.

PALABRAS CLAVE: Procesamiento de imágenes; Telemedicina; Teleoperación; Telecirugía.

SUMÁRIO

A teleoperação tornou-se uma das áreas mais prolíficas de pesquisa, devido ao grande número de aplicações em que se pode aplicar o paradigma de operação remota.

A teleoperação consiste em várias disciplinas, áreas ou técnicas, desde comunicação até controle de sistemas e desde realidade virtual até processamento de sinal digital. Assim, o progresso individual nestas áreas pode ser aplicado ao desenvolvimento de sistemas de teleoperação. Este artigo fornece uma compilação de termos e conceitos importantes apresentados na área. Também apresenta-se uma coleção dos mais recentes avanços nos sistemas de teleoperação, com especial ênfase na telesurgery e investigação nesta área nos últimos anos, fazendo uso de processamento de imagens e robótica cooperativa para aumentar a eficácia sistemas de telemedicina, a fim de aumentar a qualidade de procedimentos cirúrgicos.

PALAVRAS-CHAVE: Imagem; Telemedicina; Teleoperação; Telecirurgia.

I. INTRODUCTION

Teleoperation could be defined as a remote operation or manipulation of a system in order to accomplish some task. The word "remote" emphasizes that the controlled vehicle or system is at some distance from the operator [1]. This means that there is no direct or visual contact from the operator with the controlled system. Taking this into account, every teleoperated system is made by two fundamental components: a local component (master/ operator) and a remote one (slave). But these two systems are composed by individual modules that perform predetermined functions. A basic teleoperation system scheme is shown in Fig. 1, illustrating basic components and their functions, from which the most important are:

• Operator: the person in charge of performing the desired task. For example, in telemedicine tasks it is often a trained physician.
• Display: the way the information is presented to the operator. It is generally composed by devices that present video information such as LCD displays or head mounted displays. More recently, other types of sensory information has been added, such as audio and tactile information in order to improve the level of telepresence.
• Controls: the devices used by the operator to express the desired motion or action that the slave mechanism must perform. Usually these controls include customized hardware that take into account the kind of task the system performs. For example, in telemedicine these controls consist of devices with multiple degrees of freedom, with high motion sensing resolution and force feedback.
• Transmission: the channel whereby the information is transmitted between the local and the remote zone. It can use existing communication channels such as Ethernet, Internet, radio frequency communication, IEEE wireless communication and even satellite communication. However, the choice of the method depends on the kind of task the system will perform since critical tasks, such as telesurgery, require extremely low latencies.
• Sensors: these components are in charge of sensing the different variables or states of the remote system that include both the devices and the environment.
  In a teleoperated system they are composed by a robot.
  The sensors are usually used to measure the inner variables of the robot such as angles, distances, positions, etc. Besides those, the most common sensors used in the remote zone include the video and image sensors. These provide data directly to the data displays in the local zone.
• Actuators: the devices that perform the physical actions (such as movements) in the remote zone. Generally these actuators are electrical motors, and hydraulic and pneumatic actuators. Most recent studies have propose other kind of actuators such as magnetic ones. There have been also studies that aim to improve the performance of traditional actuators, by means of custom adaptation according to the task been performed.
• Slave controller: There exists several ways of interaction or control of the slave actions. The most common ones include:
  • Closed loop control (Direct teleoperation): The operator controls the actuators of the teleoperator by direct (analog) signals and gets real-time feedback.
  • Coordinated teleoperation: The operator controls the actuators, but now there is some internal control loop in the slave. However, there is no autonomy included in the remote side. The remote loops are used only to close those control loops that the operator is unable to control because of the delay.
  • Supervisory control: Most of the control part is to be found on the teleoperator side. The teleoperator can now perform part of the tasks more or less autonomously, while the operator mainly monitors and gives high-level commands [1].

The operator perform these interaction or control with the slave through a communication channel. Also, with a set of physical or virtual controls and a series of sensors that provide the slave with information of operator's intention. In the meantime, the slave provides information to the operator through this communication channel, a series of sensor that collect data from the environment and the slave itself and through a display that reflect all this information.

The basic system shown at Fig. 1 has been customized to fulfill specifications of new areas of application that have been emerged. Specially, in those areas were the operator is in danger performing a task or where the application of teleoperation creates significant improvements on the task performance. In this way, a rich number of possible application areas are created. For example, a well-known application of teleoperated systems include military operations such as those conducted by teleoperated drones over Afghanistan, Iraq and Pakistan [2]. Table 1 summarizes some of the most common uses of teleoperated systems.

Table 1

Is the interest of this article to provide an update survey on developments of telesurgery applications presented at [1, 3-5], that belong to a time range that comes from 2005 to 2008. Also, focus this state of the art review onto telesurgery applications that make use of machine vision and cooperative robotics. Together, these applications allow surgeons to improve the effectiveness and quality of medical procedures. Some examples are provided in [6,7] where several techniques were used to improve the quality and speed transmissions of images from telesurgery procedures. Another example was provided in [8], where instrument tracking was used to improve the awareness state of surgeons. Similarly, references [9,10] illustrate the use of 3D imaging systems to improve the diagnosis level. Finally, references [11] presents a technique that improves the control of robots that shared a task.

This article is divided into 5 sections. Section 2 shows the more recent advancements on individual components of a teleoperation systems, for example the communication and control techniques, the use of sensors, actuators and data visualization systems. Section 3 shows representative developments within teleoperation in certain application areas as telemedicine, space exploration and military operations. Section 4, presents an approach to telesurgery using machine vision and cooperative robotics techniques. The final section 5 provides a summary and conclusions of the present survey.

II. STATE OF THE ART OF TELEOPERATION SYSTEM COMPONENTS

In this section recent developments on the subsystem and module that compose teleoperation systems are presented. All application areas (medical and nonmedical) are considered. Several research studies include improvements over more than one component, but the most important contribution determines their classification.

As stated earlies (Fig. 1) there are two main zones in every teleoperated system a local zone and a remote zone. The local zone is where every data display device and every haptic and force feedback system is present. Video display is probably the most relevant form to present data to the operator but as more level of telepresence is required, other sensory channels should be included.

At Fig. 2 several components that belong to the local zone are showed, including components that have become common in this kind of application (stereoscopic displays, and controllers with force feedback). Other components, such as head mounted displays and gloves with force feedback are included, and while not as common, their inclusion in teleoperated systems, improves vastly the level of telepresence.

2.1 Data visualization and telepresence

In a teleoperated system, components for data display are a crucial part for the performance of the system. There are several kinds of components and methods for data visualization, but those related with visual data are the most important. Investigations on how important and how visual information improves task performance in teleoperated systems have been made as well as the evaluation of the influence of visual feedback on haptic exploration [12, 13]. In reference [14], investigators demonstrated how stereo vision contributes to the improvement of the perception of some depth cues. McDaniels et al studied how through a computer vision algorithm, perceptual surface roughness classification is performed, that unlike previous approaches, is designed to handle multiple roughness categories within the same image [15].

There are also research studies focusing in the improvement of the quality of the data visualized by former citations, such as [16] that presents a dual-camera vision system that allows the creation of a high-resolution image, that has seven times the resolution of images typically with commercial endoscopes. Thus, not only visual data had received the attention of researchers, but also and specially information for haptic and force feedback system implementation, because there are many different uses for it. Those include training medical practitioners, teleoperation, or navigation of virtual environments. Haptic devices can also be used to present information, and consequently, the user gains quantitative, qualitative, or holistic knowledge about the presented data [17]. At [18] a self-organizing fuzzy adaptive mapping algorithm (SOFAMap) was used to create high-fidelity haptic feedback improving the operator's awareness of unstructured environment, which was also achieved in [19], where a method that can partially cancel the haptic feedback deterioration, created by the bias portion of Wave-Variable-Based teleoperation systems was created.

Regarding force feedback, efforts to improve the accuracy of force reflection are reflected in works like [20], where a position-sensor-based force reflection framework for bilateral teleoperation systems, generated more accurate responses. In reference [21] a fuzzy force-feedback (FFF) augmentation system delivers additional information to the operator, which allows him to control smoothly a group of robots. Table 2 shows research tendencies for each sensory channel used in teleoperated systems.

Table 2

2.2 Control techniques

As aforementioned, three ways of implement the control of the remote zone of the teleoperated system. Each one of these ways has inherent variables that engineers must deal with, when in the design process, in order to have a reliable, stable and efficient control system. The most relevant variables include:

• Delay: every digital system has an inherent delay. In teleoperated systems besides the one inside digital controllers and feedback components, there exits the delay related to the communication channel and to the operator. So normal delay time in teleoperated systems depends on the distance between the local and the remote zone. In cases where the distance is higher than 300.000 km it could be always expected delays greater than 1 second.[22, 23].
• Transparency: Transparency means that the technical medium between operator and environment is not felt, i.e. that the dynamics of master and slave are canceled out [24] in a more formal way it means the equality of velocities and forces [25].
• Stability: Along with transparency, stability is a primary objective to be reached by a controller. In teleoperation system, stability not only takes into account the classic variable of the remote system (system order, natural system period) but also the inherent delays of this kind of system (transmission's delay, operator's delay).[26, 27]
• Linearity: practically every real system is nonlinear and as teleoperation systems grow in complexity the level of non-linearity also grows. Several techniques have been implemented to deal with this problem from complex adaptive algorithms to simple PID structures. Whichever the selected method, it has to take into account the delay element and the need for highly transparent systems.

Great number of investigations for efficient control techniques have been made. Some of them consider the aforementioned parameters in several different ways that range from adaptive to predictive control, and from model based to classic schemes control. It is worth mentioning that predictive and model based control can only give satisfying results if proper description of the system's behavior is provided [28]. For example, some studies such as the one reported in [29] use classical control solutions based on Kessler's Extended Symmetrical Method, which was formulated in a cascade control approach, obtaining good settling times, phase margins and overshoot percentage, for latencies as higher as 2 seconds, for a telerobotic system. Another research that also has good results eliminating overshoots is presented in reference [30], where a novel approach that combines passivity-based techniques and reset-control principles is used to control a remote crane. This operation achieved an overshoot reduction ranging from 50% to 100% reduction where there is a delay of maximum 0.5 seconds.

The importance and need of dynamic human haptic closed-loop behavior models and human perception models for the further improvement of haptic teleoperation systems has discussed [31]. Other study took into account the nature of most of this disturbances and came up with novel non-linear bilateral control schemes using the concept of “disturbance observer-based control” for nonlinear teleoperation systems [32]. With this approach, authors were able to obtain full transparency in terms of asymptotic convergence of the position tracking and force tracking errors to zero, as well as exponential convergence of disturbance tracking and position tracking errors to zero, under slow-varying disturbances. Others systems deal with non-linearity and time delays in other forms, where a first local Lyapunov-based adaptive/ nonlinear controllers are applied to linearize the system dynamics and eliminate dependency on the master and slave parameters [33]. Finally fuzzy control systems have proved a special resilience to time delays [34, 35]. Thus, studies such as the one reported in [36], use fuzzy control over teleoperated systems performing telesurgery tasks. In the study, the Tagaki-Sugeno-Kong PID-fuzzy method is used obtaining good results concerning overshoot and specially settling time.

2.3 Communication

Teleoperation means to control a system remotely in order to achieve a task. The key word here is remotely; it means that the person in charge to operate the system is not present and since he is not present, it is necessary some kind of communication channel between him and the remote system. This channel could be anyone used in modern communication systems, from any of the existing wired network topologies and wireless protocols to even satellite communication.

At the beginning of the emergence of teleoperated systems primary communication channels were simple cupper wires, where the electrical commands from the operator travels toward the controlled device. Quickly, wireless solutions (radiofrequency) gained ground, as the distance between the remote and the local zone were increasing. With the advent of space exploration the use of satellite communication became predominant in long distance applications [37]. At this point it becomes evident that communication is main cause of instability and loss of performance of teleoperated systems, resulting in the research for a control system able to handle this problem [38]. This delay extend over whichever the channels the data goes whether it be wired, wireless, satellite and more recently optic fiber, internet and sonic communication [39].

Although the main approach to handle this problem is focusing on the control mechanisms of the system, recent research also avoid this problem improving the performance of the communication systems themselves and by using predictions and statistical models of the delays. Also research has been conducted in order to reduce this effects either improving the communication methods between the local and the remote system or dealing with non-communication parameters of the system. Examples of the first class of approach includes [40] where the communication delays are assumed to be stochastically time varying and using Lyapunov methods. Findings of the study showed that the master-slave teleoperation system was stochastically stable in mean square under specific LMI conditions. A study reported in [41] reported a method to predict the time delay was created. The method was based in autoregressive models of the non-Gaussian type, and achieved a relative error below 7% under delays of 80ms.

Another technique evaluated was the IMPACT method (internal model principle and control together) which was used to handle time delays of 250 ms with an steady state error below 5% successfully [42]. Within the second class of research one the most recent proposals deal with random time delays and packet losses through the use of control strategies based on linear matrix inequalities and Markov jump linear systems [43] and time-delay compensation methods based on network disturbance and communication disturbance observer concept [44]. In the two mentioned studies, good results were obtained over time delays below 500ms. Multiple communication channels for haptic data have also been used to deal with the time delay, packet loss and jitter of a haptic data transmission system [45]. Finally, the influence of the communication delay over the performance in teleoperated systems has also been study, as in [46], where the negative effects of video transmission delays in teleoperated systems were measured quantitatively. In that study, authors also gave hints on the behavior of the ratio between haptic delay and video delay.

2.4 Actuators

Innovative research in the area of actuators deals with the development of new actuators for teleoperation tasks that overcome problems like position estimation and force transmission and control inaccuracies. Examples of these experimental actuators include investigations regarding verification of position and force transfer characteristics for flexible actuators and its application in a hand robot effector of a bilateral teleoperation system [47]. This actuator (described in Fig. 3) consists in a linear motor which pulls or pushes a thrust wire that transmits position and force to an end effector. The flexible actuator is considered to be valid for multidegree freedom of robot hands in a teleoperation system.

Usages of teleoperation systems which have been considered include the flexible actuators, telesurgery, remote manipulation in an atomic power plant, and haptic communication [48]. For these kinds of flexible actuators there are already improvements on their efficiency as described in [49], where through the use of backlash compensation, the force exerted by the actuator in order to move parts of the system was decreased. This kind of wire driven actuators could be suited in robotic Laparoscopic Surgical environments as explained in [50], where it provided benefits such as lighter weight, lower cost and faster and smoother movements (Fig. 3).

Pneumatic actuators have recently gained attention for its advantages, such as the relatively ease for force control and reliability [51]. Pneumatic actuators in a robot arm with an artificial muscle style system was built [52]. This arm achieved good level of precision and efficiency in its operation; and the system was tested over a distance of 30km between the Fukuoka Institute of Technology and the Tokyo Metropolitan College of Technology in Japan.

Taking advantage of the force control efficiency, a Stewart-Gough platform was constructed using pneumatic cylinders in order to control a submarine ROV [53]. Due to this architecture the ROV achieved a rotation speed of 8.4°/s an a linear velocity of 1.3 m/s. New forms of manipulation are appearing or becoming more common.

For example, a method to overcome the use difficulty of singularity analyses-used conventional rank determining methods for in-parallel hand platforms [54]. Another study, in [55], showed a new manipulation method based in magnetic levitation that achieved root mean square for position tracking errors which were as low as 20 to 40 µm.

Table 3 summarizes operation requirements for several applications which are compared against recent developed experimental actuators.

Table 3

III. STATE OF THE ART BY APPLICATION

This section presents representative developments in the most common application areas for teleoperated systems. Some of these developments are related to commercial systems while others are research projects. For each one advancements that have become relevant into their respective areas are discussed.

3.1 Underwater

Today ROVs probably represents the largest commercial market for mobile vehicle teleoperation [1]. This is why research have conducted to fields such as underwater wireless communication with ROV's, that has started to present good results [56].

Other fields of investigation include the control and creation of better actuators for ROV's [57-60] and the enhancement of underwater imaging (Fig. 4) [61], which is used in vision systems sensor to navigate [62]. Yet, other systems, not necessarily vision systems, are used navigate in underwater environments [63].

3.2 Space exploration

Canadarm (Fig. 5) and Robonaut2 [64] are examples of applications of teleoperation used in outer space in order to improve the performance of astronaut tasks, which previously were done by hand, exposing themselves to risks such as micro meteors or cosmic radiation.

The National Space Development Agency (NASDA) of Japan developed the Test Satellite VII manipulator that can handle delays and modeling errors. This manipulator was shown to reduce risks and cost of extra vehicular activity (EVA). Teleoperation is not solely used to avoid risk. Instead, it can also been used for more efficient exploration just like proposed at [66], where using multiple agents the surface exploration task improved greatly.

3.3 Mining

Several vehicles are being used in the mining and forestry fields because these activities are dangerous by nature. This has motivated the development of robots like the Centauroid Robot Work Partner, which was made at the Helsinki University of Technology. The Centauroid Robod was intended to address everyday life tasks in outdoor environment. Carnegie Mellon University has also developed a similar kind of robot ,but oriented to mining environments. Examples of this, is an explorer robot called Groundhogbot (Fig. 6), which can explore and take rock samples from the mine most secluded corners. And in case of an accident on a mine Cave Crawler can find an extract an injured person from the mine [1]. Finally if the need for exploration through a bore hole comes to present, reconfigurable robots like axis can be used with great reliability [67].

3.4 Telepresence

Telepresence is a concept that, means replacing the human physical presence with a robot. The robot is operated by the human operator from another location [1]. Some implementations include teleconference robots like HeadThere Robot (Fig. 7) which uses a display that can be tilted and therefore change the perspective of the teleoperator creating a more immersive experience. Another developed robot is the PEBBLES which could be used for hospitalized, homebound and special needs children and communicates via teleconferencing. PEBBLES connects children to their home classroom, allowing for total participation in education and complete social contact [68].

3.5 Military

Within this area is where the most recently developments has been made specially in the field of target acquisition, intelligence collection and reconnaissance. For example the USA military has been creating UAV's with top technology, such as synthetic aperture radar (SAR), electro-optical (EO), and infrared (IR) sensors, GPS, efficient HUD with sensory information, encrypted satellite communications between vehicles, and operator command center [69]. An example of this military technology is shown in Fig. 8.

Figure 8

Other applications have benefit from the use of state of the art materials and aerodynamic and stealthy designs. Examples include the Northrop Grumman RQ-4 Global Hawk (Fig. 9) [70], which is the most advanced UAV. Other examples include the Boeing X-45 and BAE Systems Taranis that have benefit from these newer technologies. Other field of military applications include the explosives operation (land mine detection, explosive disposal, bombs deactivation). Some example platforms the Scanjack 3500 NLD, which is a completely remote controlled mine clearing system. The experimental vehicle kutusow is a CBRNE (Chemical, biological, radiological, and nuclear) is a hazard detection robotic platform for the German Military Forces, which principal advantage is the usage of commercially available sensors that can be exchanged and upgraded easily [71].

Finally, teleoperated vehicles for both urban and nonurban search and rescue tasks have been developed for military purposes. These teleoperated vehicles occupie an important place within the newest advances [72-74] because these developments uses innovative ways to use the bots in a more efficient form. Besides, references such as [75] discuss the implementation of a data display system based in stereoscopic vision. Teleoperated robots with thermal imaging and autonomous capacities have demonostrated to be useful by allowing faster detection and rescue of people [76].

3.6 Medical

Tele-surgery is one of the first and possibly the most well-known of the applications of teleoperation to the public. Several advancements have been made in this field in areas concerning teleoperation including sensors implementation, communications optimization, input devices, augmented reality, force feedback and new forms of data visualization [77]. Of these developments, a primary example is the Da Vinci robot, shown in Fig. 10. The Da Vinci is routinely used in the clinic at some institutions worldwide and has shown to achieve operations levels which are comparable or better than those achievable by human surgeons [1-3].

For telesurgery applications, a substantial number of the investigations are focused in the development of more precise force feedback and haptic systems. For example, reference [78] presents a gripper attachment which utilizes two low-cost commercially available haptic devices to facilitate multi-point haptic grasping.

Development of dedicated actuators, surgery instrumentation and tool holders has also received great attention. For example in [79] described the development of a surgical manipulator with interchangeable workspace for minimally invasive surgery (MIS) and open surgery. Likewise, authors in [80] present a two-double parallelogram Robot with 4 degrees of freedom that eliminates deficiencies caused due to suspending massive actuators at the end effector or cabling from the base.

IV. STATE OF THE ART IN TELESURGERY

This section further elaborates in recent developments in telesurgery applications. The section emphasizes topics such as image processing, machine vision and cooperative robotics techniques. Developments in these topics are important in order to improve the quality and performance of these systems by means of enhancing the perception of the human staff (surgeon, assistants, etc.) on the several parameters from the environment and creating an efficient integration between this staff and his instrumentation.

Machine vision and image processing techniques have found an important place within teleoperation applications since its beginnings. Recent investigations have bolstered their importance by making vision systems as similar as possible to the natural human vision [81]. One example is the use of adaptive video preprocessing techniques which are used to accelerate the transmission of telesurgical video [6,7]. Other area of research efforts in making a more immersive an realistic experience for the surgeon has been made by allowing the operator to have consciousness of the instrument position at all times in MIS laparoscopic surgeries [8].

Application such as high definition 3D visualization of images has been successful to aid in procedures such as cataract surgery in pig's eyes [9]. Other authors have also concluded that for MIS, stereoscopic displays decrease surgery time while at the same time increase accuracy of surgical procedures, when the resolution of the stereoscopic displays is comparable to that of 2D displays [13].

A special excerpt for an application that is more oriented with telediagnostic and not with telesurgery is the use stereo based vision to determine the stethoscope location on the human body in a cardiac auscultation telediagnostic system [10]. It is worth mentioning that few examples of cooperative robotics within telesurgery systems have been found. A representative example is the concept of Collaborative Gaze Channeling (CGC), which was presented for improved control of surgical robots for a shared task [11]. This research also made use of machine vision algorithms in order to perform eye tracking tasks. Table 4 summarizes a brief comparison of current and former machine vision and cooperative robotics developments applied into teleoperation system is presented.

V. CONCLUSION

By studying the developments surveyed by this state of the art review, it became clear that teleoperation of systems is a topic of major relevance. It is also becoming an essential topic to certain productive areas, where it has exceeded the performance of former (and traditional) operation schemes. As such, teleoperation has become not only an important research are but also a potential source of economic growth.

Teleoperation has encountered its place within multitude of applications. Teleoperation is playing a major role with tasks that have an inherent or implicit danger for the operator. These tasks include some such as hazardous materials disposal, explosive deactivation, search and rescue. Notable, such uses are not limited to military operations, and in many cases, efforts come from both commercial and academic researchers.

This review identified two research tendencies on teleoperation applications. The first tendency does search for an increase in the autonomy and intelligent behavior through machine learning and adaptive control techniques of the remote part of the system. Hence, making that the human operator has only to operate and be aware of high level functions of the system. On the other hand, a second major tendency focuses on applications where the operator has to deal with an increase amount of information from the teleoperated environment. These multiple inputs come through haptics, visual and force feedback and data visualization systems.

In the case of telesurgery applications, there exists a substantial amount of research studies on the integration of machine vision, and several studies have been made. However, these studies are still insufficient to describe all the possible advances that both areas could generate on the perception of the operator (surgeon) from the environment and the subject submitted to the procedure: and the data that machine vision could extract from apparent non-relevant sensory information. Furthermore, cooperative robotics has not been subject to a limited number of research studies hence there is the potential that the are can still improve and extend the performance and capabilities of telesurgery systems operated under multirobot environments. This situation allows concluding that more research in these areas is still needed in order to further improve awareness of operators on the environment, quality of data augmentation incoming from vision sensors, quality and speed of image transmission and automated detection of interest features. The use of techniques from cooperative robotics is useful under environments where several robotic manipulators must accomplish an specific task thus implying that certain level of autonomy is needed.

REFERENCES

[1]. Lichiardopol S.A. Survey on Teleoperation. Technische Universitat Eindhoven , DCT report, December 2007.         [ Links ]

[2]. Pakistani intelligence officials say US missile strike kills 4 militants near Afghan border. Consulted on June 2012. Available at http://www.washingtonpost.com/world/asia_pacific/pakistansays-militants-who-crossed-over-from-afghanistan-kill-13-troopsbeheading-7-of-them/2012/06/26/gJQACVII3V_story.html.         [ Links ]

[3]. De Barros P., Lindeman, R.A. Survey of User Interfaces for Robot Teleoperation. Worcester polytechnic institute. Computer Science Technical Report Series, October 2008.         [ Links ]

[4]. Tomioka Y., Aida N., Kakehi K., Nagami K., Juzoji, H., Nakajima, I. Recent survey on patent applications for medical communications and telemedicine in Japan, U.S.A. and Europe. Enterprise networking and Computing in Healthcare Industry. HEALTHCOM. Proceedings of 7th International Workshop. 79-82, 23-25, June 2005.         [ Links ]

[5]. Kumar S. Telesurgery: An Audit. Telesurgery, 179-184, 2008.         [ Links ]

[6]. Jian X., Sclabassi R.J., Qiang L., Chaparro L.F., Marchessault R., Mingui S. Human Perception based Video Preprocessing for Telesurgery. Proceedings 29th Annual International Conference of the EMBS/IEEE, 3086-3089, 22-26, Aug. 2007.         [ Links ]

[7]. Nouri N., Abraham D., Moureaux J.-M., Dufaut M., Hubert, J., Perez M. Subjective MPEG2 compressed video quality assessment: Application to Tele-surgery. Proceedings of 2010 IEEE International Symposium on Biomedical Imaging: from nano to macro. 764-767, 14-17 April 2010.         [ Links ]

[8]. Voros S., Haber G.-P., Menudet J.-F., Long J.-A., Cinquin P. ViKY Robotic Scope Holder: Initial Clinical Experience and Preliminary Results Using Instrument Tracking. Mechatronics, IEEE/ASME Transactions 15(6), 879-886, Dec. 2010.         [ Links ]

[9]. Yoneda T., Kani K., Tabuchi A., Hiyane I., Araki K., Nakamura T., Hoshi, K. Telemedicine system using 3-D high definition image. 2007. CME 2007. IEEE/ICME International Conference, 1120-1123, 23-27 May 2007.         [ Links ]

[10]. Kumpituck S., Kongprawechnon W., Kondo T., Nilkhamhang I., Tungpimolrut K., Nishihara A. Stereo based vision system for cardiac auscultation tele-diagnostic system. ICCAS-SICE, 2009, 4015-4019, 18-21, Aug. 2009.         [ Links ]

[11]. Kwok, K., Sun, L., Mylonas, G., James, D., Orihuela-Espina, F., Yang, G. Collaborative Gaze Channelling for Improved Cooperation During Robotic Assisted Surgery. Annals of Biomedical Engineering, April 2012.         [ Links ]

[12]. Plaisier M.A., Kappers A.M.L., Tiest W.M.B., Ernst M.O. Visually Guided Haptic Search. Haptics, IEEE Transactions, 3(1), 63-72, Jan.-March 2010.         [ Links ]

[13]. Van Beurden M., IJsselsteijn W., Juola J. Effectiveness of stereoscopic displays in medicine: A review. 3D Research, 3 1-13, March 2012.         [ Links ]

[14]. Livatino S., Muscato G., Privitera F. Stereo Viewing and Virtual Reality Technologies in Mobile Robot Teleguide. Robotics, IEEE Transactions, 25(6), 1343-1355, Dec. 2009.         [ Links ]

[15]. McDaniel T.L., Panchanathan S. Perceptual Surface Roughness Classification of 3D Textures Using Support Vector Machines, Haptic, Audio and Visual Environments and Games. HAVE 2007. IEEE International Workshop, 154-159, 12-14, Oct. 2007.         [ Links ]

[16]. Tonet O., Focacci F., Piccigallo M., Mattei L., Quaglia C., Megali G., Mazzolai B., Dario, P. Bioinspired Robotic Dual-Camera System for High-Resolution Vision. Robotics, IEEE Transactions on, 24(1), 55-64, Feb. 2008.         [ Links ]

[17]. Paneels S., Roberts J.C. Review of Designs for Haptic Data Visualization. Haptics, IEEE Transactions, 3(2), 119-137, April-June 2010.         [ Links ]

[18]. Linda O., Manic M. Self-Organizing Fuzzy Haptic Teleoperation of Mobile Robot Using Sparse Sonar Data. Industrial Electronics, IEEE Transactions, 58 (8) 3187-3195, Aug. 2011.         [ Links ]

[19]. Yongqiang Y., Liu, P.X. Improving Haptic Feedback Fidelity in Wave-Variable-Based Teleoperation Orientated to Telemedical Applications. Instrumentation and Measurement, IEEE Transactions, 58(8), 2847-2855, Aug. 2009.         [ Links ]

[20]. Keehoon K., Wan K.C., Cavusoglu M.C. Description of Instantaneous Restriction Space for Multi-DOFs Bilateral Teleoperation Systems Using Position Sensors in Unstructured Environments. Robotics, IEEE Transactions, 25(5), 1150-1158, Oct. 2009.         [ Links ]

[21]. Linda O., Manic M. Fuzzy Force-Feedback Augmentation for Manual Control of Multirobot System. Industrial Electronics, IEEE Transactions, 58 (8) 3213-3220, Aug. 2011.         [ Links ]

[22]. Aziminejad M., Tavakoli R.V., Patel M. Moallem Stability and performance in delayed bilateral teleoperation: Theory and experiments. Control Engineering Practice, 16, 1329-1343, 2008.         [ Links ]

[23]. Chopra N., Berestesky P., Spong M.W. Bilateral teleoperation over unreliable communication networks. Control Systems Technology, IEEE Transactions 16, 304-313, 2008.         [ Links ]

[24]. Teleoperation. Available at http://autsys.tkk.fi/fsr/attach/Material/Teleoperation.pdf. Consulted on June 2012.         [ Links ]

[25]. Passenberg C., Peer A., Buss M. A survey of environment-, operator-, and task-adapted controllers for teleoperation systems. Mechatronics, 20(7), 787-801. October 2010.         [ Links ]

[26]. Lawrence D. Stability and transparency in bilateral teleoperation. IEEE Trans Robot Autom. 9 624-637, 1993.         [ Links ]

[27]. Arcara P., Melchiorri C. Control schemes for teleoperation with time delay: A comparative study. Robotics and Autonomous Systems, 38(1), 49-64. January 2002.         [ Links ]

[28]. Lee D., Li P.Y. Passive bilateral feedforward control of linear dynamically similar teleoperated manipulators. IEEE Transactions on Robotics and Automation, 19, 443-456, 2003.         [ Links ]

[29]. Haidegger T., Kovacs L., Precup E., Preitl S., Benyo B., Benyo Z. Cascade Control for Telerobotic Systems Serving Space Medicine. 18th IFAC World Congress. Milano, Italy, Aug. 2011.         [ Links ]

[30]. Villaverde A.F., Blas A.B., Carrasco J., Torrico A.B. Reset Control for Passive Bilateral Teleoperation. Industrial Electronics, IEEE Transactions, 58(7), 3037-3045, July 2011.         [ Links ]

[31]. Hirche S., Buss M. Human-Oriented Control for Haptic Teleoperation. Proceedings of the IEEE, 100(3), 623-647, March 2012.         [ Links ]

[32]. Mohammadi A., Tavakoli M., Marquez H.J. Disturbance observer-based control of non-linear haptic teleoperation systems. Control Theory & Applications, IET, 5(18), 2063-2074, December 8, 2011.         [ Links ]

[33]. Shahdi A., Sirouspour S. Adaptive/Robust Control for Time-Delay Teleoperation. Robotics, IEEE Transactions, 25(1), 196-205, Feb. 2009.         [ Links ]

[34]. Ottensmeyer M. P., Hu J., Thompson J. M., Ren J., Sheridan T. B. Investigations into performance of minimally invasive telesurgery with feedback time delays. Presence Teleop. Virtual Environ., 9(4), 69-382, 2000.         [ Links ]

[35]. Farkhatdinov J. H., Ryu J., Poduraev. Rendering of environmental force feedback in mobile robot tele-operation based on fuzzy logic. Proc. of IEEE International Symposium on Computational Intelligence in Robotics and Automation. Korea, 503-508, 2009.         [ Links ]

[36]. Precup R.-E., Kovacs L., Haidegger T., Preitl S., Kovacs A., Benyo B., Borbely E., Benyo Z. Time delay compensation by fuzzy control in the case of master-slave telesurgery. Applied Computational Intelligence and Informatics (SACI), 2011. 6th IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI), 305-310, 19-21, May 2011.         [ Links ]

[37]. Vertut Jean, Coiffet Philippe, 1985. Teleoperation and Robotics, Springer, ISBN: 0137821948.         [ Links ]

[38]. Bratt M. Teleoperation with significant dynamics. PhD Thesis, Computer Science and Communication, KTH Sweden, 2009.         [ Links ]

[39]. Ferworn A., Roque R., Vecchia I. MAX: Wireless Teleoperation via the World Wide Web, Network-Centric Applied Research Team (N-CART). Ryerson Polytechnic University, May 2004.         [ Links ]

[40]. Kang Y., Li Z., Shang W., Xi H. Motion synchronisation of bilateral teleoperation systems with mode-dependent time-varying communication delays. Control Theory & Applications, IET, 4(10), 2129-2140, October 2010.         [ Links ]

[41]. Hu T.J., Huang X.X., Tan Q. Time delay prediction for space teleoperation based on Non-Gaussian Auto-Regressive Model, Modelling, Identification & Control (ICMIC). Proceedings of International Conference, 567-572, 24-26, June 2012.         [ Links ]

[42]. Denasi A., Kostic D., Nijmeijer H. Time Delay Compensation in Bilateral Teleoperations Using IMPACT. Control Systems Technology, IEEE Transactions. 21(3), 704-715. 2013.         [ Links ]

[43]. Walker K.C., Ya-Jun Pan, Gu J. Bilateral Teleoperation Over Networks Based on Stochastic Switching Approach. Mechatronics, IEEE/ASME Transactions, 14(5) 539-554, Oct. 2009.         [ Links ]

[44]. Natori K., Tsuji T., Ohnishi K., Hace A., Jezernik K. Time-Delay Compensation by Communication Disturbance Observer for Bilateral Teleoperation Under Time-Varying Delay. Industrial Electronics, IEEE Transactions, 57(3), 1050-1062, March 2010.         [ Links ]

[45]. Suzuki N., Katsura, S. Data transmission with multiple-routes for wireless haptic communication system. Advanced Motion Control (AMC), 12th IEEE International Workshop, 1-6, 25-27 March 2012.         [ Links ]

[46]. Hyuk K., Jee-Hwan R. A study on the effect of haptic to video time-delay on teleoperation and a comment for improving the performance. Control, Automation and Systems (ICCAS) 11th International Conference, 1329-1332, 26-29 Oct. 2011.         [ Links ]

[47]. Hyodo S., Soeda Y., Ohnishi K. Verification of Flexible Actuator From Position and Force Transfer Characteristic and Its Application to Bilateral Teleoperation System. Industrial Electronics, IEEE Transactions, 56(1), 36-42, Jan. 2009.         [ Links ]

[48]. Mitsantisuk C., Ohishi, K., Katsura, S. Compensation of backlash for improving the efficiency of flexible actuator in bilateral teleoperation system. IECON - 37th Annual Conference on IEEE Industrial Electronics Society, 4220-4225, 7-10 Nov. 2011.         [ Links ]

[49]. Direkwatana C., Suthakorn J., Wilasrusmee C. Development of wire-driven laparoscopic surgical robotic system, MULapaRobot. Robotics and Biomimetics (ROBIO), IEEE International Conference, 485-490, 7-11 Dec. 2011.         [ Links ]

[50]. Ponomareva E. Hydraulic and Pneumatic Actuators and their Application Areas. Mechatronics - Foundations and Applications, TU München, May 2006.         [ Links ]

[51]. Kato T., Higashi T., Shimizu K. Teleoperation of a Robot Arm System Using Pneumatic Artificial Rubber Muscles: Teleoperation over the Internet Using UDP and a Web Camera. Broadband, Wireless Computing, Communication and Applications (BWCCA), International Conference 714-718, 4-6 Nov. 2010.         [ Links ]

[52]. Saltaren R., Garcia C.E., Alvarez C., Aracil R. Experiences and results from designing and developing a 6 DoF underwater parallel robot. Robotics and Autonomous Systems, 59(2), February 2011, 101-112, ISSN 0921-8890, 10.1016/j.robot.2010.10.005.         [ Links ]

[53]. Collins C.L., Long G.L. The singularity analysis of an in-parallel hand controller for force-reflected teleoperation. Robotics and Automation, IEEE Transactions 11(5), 661-669, Oct. 1995.         [ Links ]

[54]. Mehrtash M., Tsuda N., Khamesee M.B. Bilateral Macro-Micro Teleoperation Using Magnetic Levitation, Mechatronics, IEEE/ ASME Transactions, 16(3), 459-469, June 2011.         [ Links ]

[55]. Law E., Bradbeer R., Yeung L, Bin L., Zhongguo G. Communication with an underwater ROV using ultrasonic transmission. Mechatronics and Machine Vision in Practice, Chiang Mai, Thailand, 295-304, September 2002.         [ Links ]

[56]. Atlas Hybrid Manipulator. Consulted on June 2012. Available at http://www.oceaneering.com/rovs/rov-technologies/atlas-hybrid-manipulator/.         [ Links ]

[57]. Yanfu L., Qifeng Z., Xisheng F. Inverse kinematic solution and simulation for ROVs-Manipulators system. Control and Decision Conference, 2008. 3345-3348, 2-4 July 2008.         [ Links ]

[58]. Hildebrandt M., Christensen L., Kerdels J., Albiez J., Kirchner F. Realtime motion compensation for ROV-based tele-operated underwater manipulators. OCEANS - EUROPE, 1-6, 11-14 May 2009.         [ Links ]

[59]. Jianjun Y., Liquan W., Peng J., Zhuo W. Development of a 7-function hydraulic underwater manipulator system. Mechatronics and Automation. International Conference, 1202-1206, 9-12 Aug. 2009.         [ Links ]

[60]. Kocak D., Dalgleish F., Caimi F., Schechner Y. A Focus on Recent Developments and Trends in Underwater Imaging. Marine Technology Society Journal, 42(1), 52-67, Spring 2008.         [ Links ]

[61]. Jordán M.A., Berger C.E., Bustamante J.L., Hansen S. Path tracking in underwater vehicle navigation: on-line evaluation of guidance path errors via vision sensor. Decision and Control (CDC). 49th IEEE Conference, 2517-2522, 15-17 Dec. 2010.         [ Links ]

[62]. Karras G.C., Loizou S.G., Kyriakopoulos K.J. Semi-autonomous teleoperation of a non-holonomic underwater vehicle using a Laser Vision System: A visual-servoing switching control approach. Control and Automation. 17th Mediterranean Conference, 797-802, 24-26 June 2009.         [ Links ]

[63]. Diftler M.A., Ahlstrom T.D., Ambrose R.O., Radford N.A., Joyce C.A., De La Pena N., Parsons A.H., Noblitt A.L. Robonaut 2 -Initial activities on-board the ISS. Aerospace Conference. IEEE, 1-12, 3-10 March 2012.         [ Links ]

[64]. Woo-Keun Y., Goshozono T., Kawabe H., Kinami M., Tsumaki Y., Uchiyama M., Oda M., Doi T. Model-based space robot teleoperation of ETS-VII manipulator. Robotics and Automation, IEEE Transactions, 20(3), 602-612, June 2004.         [ Links ]

[65]. Atherton J.A., Hardin B., Goodrich M.A. Coordinating a Multi-Agent Team Using a Multiple Perspective Interface Paradigm. Proceedings of the AAAI Spring Symposium: To Boldly Go Where No Human-Robot Team Has Gone Before. Stanford University, California, 47-51. 2006.         [ Links ]

[66]. Morris A., Ferguson D., Omohundro Z., Bradley D., Silver D., Baker C., Thayer S., Whittaker C., Whittaker W. Recent developments in subterranean robotics. Journal of Field Robotics. 23(1), 35-57, January 2006.         [ Links ]

[67]. Providing education by bringing learning environments to students. Consulted on June 2012. Available at http://www.telbotics.com/background.htm.         [ Links ]

[68]. Unmanned aircraft system roadmap 2005-2030. Consulted on June 2012. Available at http://www.fas.org/irp/program/collect/uav_roadmap2005.pdf.         [ Links ]

[69]. U.S army roadmap for UAS 2010-2035. Consulted on June 2012. Available at http://www-rucker.army.mil/usaace/uas/US%20Army%20UAS%20RoadMap%202010%202035.pdf.         [ Links ]

[70]. Rohling T., Bruggemann B., Hoeller F., Schneider F.E. CBRNE hazard detection with an Unmanned vehicle, Safety. Security & Rescue Robotics (SSRR). 2009 IEEE International Workshop, 1-5, 3-6 November 2009.         [ Links ]

[71]. Casper J.L., Murphy R.R. Workflow study on human-robot interaction in USAR, Robotics and Automation, 2002. Proceedings. ICRA IEEE International Conference 2 1997-2003.2, 2002.         [ Links ]

[72]. Cooper J., Goodrich M. A. Towards Combining UAV and Sensor Operator Roles in UAV-Enabled Visual Search. Proceedings of ACM/IEEE International Conference on Human-Robot Interaction, March 2008, Amsterdam, The Netherlands, 351-358.         [ Links ]

[73]. Doroodgar B., Nejat G. A hierarchical reinforcement learning based control architecture for semi-autonomous rescue robots in cluttered environments. Automation Science and Engineering (CASE), IEEE Conference, 948-953, 21-24 Aug. 2010.         [ Links ]

[74]. Martins H., Ventura R. Immersive 3-D teleoperation of a search and rescue robot using a head-mounted displa. Emerging Technologies & Factory Automation (ETFA). IEEE Conference, 1-8, 22-25 Sept. 2009.         [ Links ]

[75]. Mano H., Kon K., Sato N., Ito M., Mizumoto H., Goto K., Chatterjee R., Matsuno F. Treaded control system for rescue robots in indoor environment. Robotics and Biomimetics (ROBIO) IEEE International Conference, 1836-1843, 22-25 Feb. 2009.         [ Links ]

[76]. Quan-Zen A., Horan B., Najdovski Z., Nahavandi S. Enabling multipoint haptic grasping in virtual environments, 3D User Interfaces (3DUI). IEEE Symposium, 55-58, 19-20 March 2011.         [ Links ]

[77]. Kim K., Song H., Suh J., Lee J. A Novel Surgical Manipulator with Workspace-Conversion Ability for Telesurgery. Mechatronics, IEEE/ ASME Transactions, 99, 1-12, September 2011.         [ Links ]

[78]. Hadavand M., Mirbagheri A., Salarieh H., Farahmand F. Design of a force-reflective master robot for haptic telesurgery applications: RoboMaster1. Engineering in Medicine and Biology Society (EMBC) Annual International Conference of the IEEE, 7037-7040, Aug. 30-Sept. 3, 2011.         [ Links ]

[79]. Kovacs L., Haidegger T., Rudas I. Surgery from a distance-Application of intelligent control for telemedicine, Applied Machine Intelligence and Informatics (SAMI), IEEE 11th International Symposium, 125, 129, Jan. 31 2013-Feb. 2 2013doi: 10.1109/SAMI.2013.6480959.         [ Links ]

[80]. Wang M.-L., Wu J.-J., Lee P.-Y., Hu M.-H., Kumar A., Chen L.-X., Liu K.-C., Marescaux J., Nicolau S., Vemuri A., Soler L. A landmark based registration technique for minimally invasive spinal surgery, Consumer Electronics (ISCE), IEEE 17th International Symposium, 235, 236, 3-6 June 2013.         [ Links ]