Robots for search site monitoring, suspect guarding, and evidence identification

Received Dec 27, 2019 Revised Jan 12, 2020 Accepted Mar 4, 2020 As an initial trial and in response to a lack of technological applications in government agencies, we have developed three multifunctional robots in accordance with the work environment and the nature of our tasks. Search site monitoring robot is fitted with a panoramic camera and large wheels for walk-around search site monitoring. Suspect guarding robot follows and guards a suspect by tracking an augmented reality marker worn by the suspect and identifying the human body through an infrared thermal camera. For the evidence identification robot, You Only Look Once (YOLO) is utilized to identify some specific evidence on search site and is equipped with a carrier and a high-torque motor for evidence transportation; it is set to issue warnings and emails to relevant personnel on specific emergencies. We have performed multiple experiments and tests to confirm the robots’ effectiveness, verifying their applicability of technological task support in government agencies.


INTRODUCTION
Technological evolution has brought about convenience in our daily life with extremely rapid pace in various manners, one of which is the widespread application of robots in various fields, such as living cleaning [1], metal [2] and odor [3] detection, cultural preservation [4], and even paddy raking [5]. However, robots have rarely been employed by government agencies. Compared to civilian enterprises, government agencies are characterized by stiff and inflexible organizational structures, which have limited innovative technology application. The overall goal of this study is to promote the use of technology in government affairs, and thus three multifunctional robot, in the context of the work environment and task characteristics of the bureau, have been developed focusing on environmental monitoring, individual following, and object identification, respectively.
Methods developed for remote monitoring have been diversified; in addition to basic visual feedback, numerous functions have been devised to serve the goals of development. Salh et al. employed an artificial neural network in a field-programmable analog array to control robot actions, applied feature extraction algorithm for facial identification, and implemented an MQ4 sensor coupled with a peripheral circuit to create a smart monitoring robot capable of detecting flammable gas [6]. With the goal of applying a network-based robot system in remote monitoring, Sundaram [9]. In response to the rising awareness of safety issues worldwide, Rashid et al. designed a Raspberry Pi-based mobile monitoring system for live streaming and voice control; the system enabled remote monitoring system control through dual-tone multifrequency(DTMF) control using a network interface or a mobile phone keyboard [10]. Individual following requires locking onto a specifically targeted individual person without confusing that person with other individuals. An unmanned ground vehicle can be equipped with a video camera or other types of sensors to detect and track individuals within its range of sight [11]. Most studies have focused on refining the existing technology or proposing solutions to difficulties encountered in practical applications, and most have employed sensors for obtaining distance information. Wang et al. used an extended Kalman filter along with data collected by camera and a supersonic wave sensor to develop a real-time three-dimensional (3D) individual-tracking system; they attempted to overcome its practical problems (such as occlusion) and to improve the scale precision of 3D data [12]. To solve the disadvantages of robots that track users from the back, Nikdel et al. developed one that follows its user from the front; an extended Kalman filter, camera with different fields of view and a laser rangefinder were employed to estimate the user's relative position and speed, and a preestablished occupancy grid map was implemented to detect the target and predict its actions and trajectory of movement [13]. Using the A* path planning algorithm and data obtained through light detection and ranging (lidar) and gyroscopes, Huskić et al. developed a tracking robot that can follow targets while moving across various types of terrain and dynamic environments at high speeds [14]. Chen et al. improved tracking quality through an consensus of corresponding methods and image preprocessing technology; they employed supervised learning to update features for redirecting processes for redetection or complex backgrounds, developing FOLO, a two-dimensional appearance-based tracking robot [15]. Chen et al. employed two methods to develop automatic tracking robots [16,17]. Selected online ada-boosting(SOAB) was integrated with depth information to improve upon the inability of the online ada-boosting(OAB) algorithm to maintain a fixed target size in a changing environment [16]. The RGB channel and computed stereo depth image (called RGB-stereo depth, RGB-SD) were entered into a convolutional neural network (CNN) to output required information, and a proportional-integral-derivative (PID) controller was employed to control the robot in target tracking; the latest action and posture of the target individual were used to calculate and predict its path in response to its temporary disappearance from the sight [17]. To improve upon the existing visual-based human-tracking method, Gupta et al. employed Speeded Up Robust Features (SURFs) for target tracking; a k-dimensional tree (K-D tree) was jointly exploited with a Kalman filter for data classification to detect changes in posture, and a servo controller was applied to command the robot to follow the target [18].
Object identification is the most developed field in deep learning; it enables a system to establish and train a model according to various needs and thus is applied for diverse purposes. The Convolutional Neural Network (CNN) paradigm was created to improve upon the ability of deep neural networks to process only one-dimensional data [19] and is one of the most frequently applied technology paradigms for image feature extraction. Several CNN-based models have been established, such as Regions with CNN (R-CNN), Region Proposal Network (RPN), and You Only Look Once (YOLO) [20]. Yu et al. applied object identification technology in an existing advertisement system to improve outdoor advertising efficacy. They proposed an audience-oriented targeted advertising system integrated with biostatistics and machine learning; Microsoft Face application programming interface (API) was used to identify the sex and age of an individual, and a Single Shot MultiBox Detector (SSD) was employed to attain identification of multiple objects including vehicles, and the identification results were then used to determine the types of advertisements to broadcast [21]. In response to the problems associated with increasing traffic and its dynamic nature, Iyer et al. exploited a SSD to develop a traffic signal system able to adjust in real time. The types and number of vehicles at each intersection were detected and counted to calculate the duration of a follow-up green light session and the time until the next green light session. Moreover, after each cycle, in which all intersections had undergone a green light session, the signal system automatically adjusts itself according to the current traffic situations for considerably higher efficiency than conventional signal systems with fixed-time traffic signals [22].
Hardware of the three multifunctional robots proposed in this study is all based on TurtleBot3 and is intended for search site monitoring, suspect guarding, and evidence identification, respectively. The site-monitoring robot is equipped with a panoramic camera and the large wheels enable its high mobility to walk around and monitor the search sites through live streaming; thus users are able to view the site remotely through a corresponding software application. We explicated an automatic tracking technology to have the suspect-guarding robot to follow a suspect to be interrogated; a laser rangefinder and an augmented reality (AR) marker are employed to follow the suspect, and a thermal camera is applied to identify the suspect. As for the object identification robot, YOLO was applied to identify evidence at the search site and is fitted with a carrier for transporting the evidence found; it also issues warnings when anomalies occur.

RESEARCH METHOD
This study developed several robotic technologies independently using the Robot Operating System (ROS) [23]. The ROS provides most of the functions of traditional operating systems such as hardware layer abstraction, low-level equipment control, interprocess message transmission, and package management. Additionally, relevant tools and procedural libraries are provided that can be used to acquire, compile, and edit code and achieve distributed computing. The ROS standard package provides various stable and adjustable robot algorithms. The standardized ROS communication interface means that developers can devote more time on design and actualization of new ideas and computations, thereby avoiding repetition of existing research outcomes. Modern robots usually require multiple computers to calculate the numerous processes they conduct. Thus, a robot can be equipped with several computers, with each computer powering a part of the robot's transducer and driver. Alternatively, users can send control commands to a robot through their computers, such as a tablet or smartphone. This type of human-machine interactive interface can be considered as part of a distributed system. Therefore, the ROS can help resolve communication problems that arise between different processes when several computers are part of a distributed system. Based on the ROS, we developed functions such as remote monitoring, automatic individual following and object identification; the design and implementation of each function was as follows.

Autonomous smart navigation 2.1.1. Mapping
High-precision Lidar (Figure 1) was used to construct a customized map (Figure 2) of the building using the gmapping algorithm [24]. The Rao-Blackwellized particle filter was used with the gmapping algorithm to achieve simultaneous localization and mapping (SLAM). A study [25] indicated that gmapping has high stability and excellent performance in terms of the error rate and CPU load.

Positioning
Taking the data from the Lidar and an inertial measurement unit (Figure 3), the adaptive Monte Carlo localization (AMCL) algorithm [26] was adopted to achieve positioning (Figure 4). The customized map was used with the algorithm to dynamically construct probability distributions of particles. Then, the Lidar-measured values were used to adjust the probability distributions until the positioning results converged.

Route planning and following
The probabilistic roadmap (PRM) algorithm [27] was used for route planning by constructing connections between nodes that were subsequently used to locate obstacle-free routes between the starting and finishing point ( Figure 5). The PurePursuit algorithm [28] was used to execute the planned route, and look-ahead points were adjusted to ensure the route was smoothly and correctly taken ( Figure 6).

Dynamic environment detection and obstacle avoidance
The vector field histogram(VFH+) algorithm [29,30] was adopted for dynamic environment detection and obstacle avoidance ( Figure 6). This algorithm used the data received from the sonar (Figure 7) and Lidar (Figure 1) to construct the polar histogram of obstacles. Subsequently, the histogram thresholds and minimum turning radium were used to determine the required route for obstacle avoidance (Figure 8).

Remote human-machine control interface
The representational state transfer (RESTful) API [31,32] (Figure 9) not only enabled us to operate intelligent machines on websites, applications, and mobile devices, but sent images from its visual system to users ( Figure 10). a. The RESTful API comprises three elements [33]: -A URL for the web service (e.g., http://example.com/resources/). -A data-interchange format that is accepted and returned by the web service (e.g., JSON).
-RESTful methods for making requests that are supported by the web service (e.g., POST, GET, PUT, or DELETE). b. The RESTful API uses HTTP as the underlying protocol [32,34]. Compared with conventional web services, RESTful is lightweight with both client and server sides. On the client side, HTTP is used to request resources from the server side. The server side is responsible for processing requests and allocating resources. HTTP operation that can be used on websites, applications, and mobile devices enables quick and simple operation of smart machines using a visual interface. Figure 9. RESTful API framework Figure 10. Human-machine remote control interface (mobile devices such as tablets, smart phones…etc.)

Mobile search site monitoring
The search site monitoring robot, equipped with a panoramic camera and large wheels, monitors search sites while navigating by random walk methods. The GV-VR360 panoramic camera ( Figure 11) provides an all-around perspective and supports multiple functions such as live streaming. In practice, a single search mission can be conducted at multiple locations. By default, the search site monitoring robot may be applied outdoors or at sites without wireless networks, and in scenarios where navigation maps cannot be illustrated in advance. We replaced the small wheels in TurtleBot with larger ones and thus the robot can adapt to various road surfaces and terrain types; a 4G network card and a gateway are installed for a virtual private network (VPN) connection, enabling remote control of the robot while the robot moves automatically. Moreover, the robot is connected to a self-established cloud system to provide real-time video monitoring so that users are able to monitor the search status at each location and direct or adjust the search mission at any time.

Individual-following using an AR marker
An AR marker is employed to follow an individual. The Automatic Parking Vision, one of the applications in TurtleBot3, is originally designed to achieve automatic parking by tracking AR markers through a Raspberry Pi camera [35]. Because of the simple environment at the bureau and the unique purpose of our application, we had an AR marker worn by the suspect for the robot to track. In addition, to prevent the suspect from taking off the marker autonomously, an infrared thermal camera ( Figure 12) is exploited to help the robot to identify the human body through thermography. A laser rangefinder is used to measure the distance between the suspect and the robot, and Random Forest are employed to assess the location of the suspect. If the suspect leaves the range of the robot's vision for 3 seconds, the monitoring platform will issue a warning to alert users to respond immediately to the emergency, thereby achieving suspect guarding. Figure 11. GV-VR360 IP video camera Figure 12. Optris PI 230 infrared thermal camera

Object identification and warning
A Horned Sungem vision kit, a Raspberry Pi camera, and YOLO are applied for object identification. YOLO, which converts the task of object identification to a regression problem and integrates the operational procedure in a single neural network, requires relatively little calculation, is easy to train, and is fast [20,36]. Lidar and a high-torque motor are implemented for the robot to identify specific objects relevant to the focal points of the search mission; a carrier is installed for transporting the evidence found. The objects can be identified in our study now include individuals, computers, screens, keyboards, mice, backpacks, handbags, and suitcases. A corresponding monitoring platform is also installed; when a single specific object of interest is identified, the platform issues a warning, and the warning light turns red. When multiple objects of interest are detected (e.g., individuals and doors), a runaway alert is issued; when individuals and the aforementioned objects of interest are detected at the same time, an evidence destruction warning is issued; when multiple individuals are detected, a collusion warning is issued, and the warning light turns red.

RESULTS
The three multifunctional robots developed to assist the tasks of the bureau are depicted as follows.

Search site monitoring robot
Search site monitoring robot consists of a GV-VR360 panoramic camera along with other peripheral devices ( Figure 13). This robot is capable of walk-around monitoring with a large range ( Figure 14) and provides live streaming through a cloud system (Figure 15) where the videos captured by the robot are stored in the cloud for playback and examination afterwards. An application ( Figure 16) is developed as well for users to monitor the search mission at all time.  Figure 16. Interface of the application

Suspect guarding robot
Suspect guarding robot consists of a laser rangefinder, an infrared thermal camera, a Raspberry Pi camera, and other peripheral devices ( Figure 17). In our design, an AR marker was placed on the suspect's foot for the robot to track ( Figure 18). Figure 19 illustrates the interface of the robot's operational platform. On the left the following path of the robot is showed as the blue curve where the blank region indicates the amount of idle time. Table 1 lists the rate of the robot's successful tracking of the AR marker for each set of distances (50 tests per set). The robot was set to stop when it was within 30 cm of the suspect; when the target was lost for more than 3 s, the robot issued a warning and sent emails to the personnel.

Evidence identification robot
Evidence identification robot comprises a Horned Sungem vision kit, a Raspberry Pi camera, lidar, and other peripheral devices (Figure 20). Figure 21 depicts the operational platform of the robot. The map on the left shows the robot's following path, and the lights on the right indicate the objects and actions identified. Table 2 lists the robot's identification results conducted with different samples of each type of objects including both actual entities and images.

CONCLUSION
In this study three key tasks were selected as the purposes of the three multifunctional robots, and experiments and adjustments were conducted to verify the robots' feasibility and practicality. Because government action can infringe on citizens' rights, the designs of technological applications must emphasize data protection in addition to efficacy and efficiency. Nevertheless, because of the relatively simple environments and routine missions of government agencies, appropriate technological task support is desirable. In the future, self-developed training models should be adopted to improve the robots' object identification efficacy, and the promotion for the technological applications in government agencies should continue.