Fuzzy neuro-genetic approach for feature selection and image classification in augmented reality systems

ABSTRACT


INTRODUCTION
In the recent years, the researchers in the area of image processing, computer vision and robotics have shifted from 2 Dimensional technologies to 3 Dimensional technologies for capturing and analyzing images due to the recent advance in computing with respect to memory and processing power. An augmented reality system needs to perform many operations in real time including image capturing using input sensors, processing the images and providing suitable response to the user. Research in robotics and augmented reality systems use the information technology for providing efficient solution in medical diagnosis, military applications, online education, robotics and computer aided manufacturing.
Augmented reality helps to enhance the view of real world objects present in the physical environment in terms of virtual elements so that the view is nothing but the blend of real and virtual elements [1]. It is interactive in nature and hence it lives between the reality and the virtual world. Using virtual reality the simulation of real world is possible and it also provides additional features for motion, sizing, coloring and effective recognition of objects. In computer vision based robotics, two dimensional images were used for forecasting the next move of the robot by analyzing the past and the current moves of the robot along with the rules governing change of the environment by applying constraint satisfaction techniques. On the other hand in augmented reality, the vision recognition system of a robot uses a three dimensional rendering engine for blending the virtual objects with the real objects in the world [2]. Learning is an important phenomenon in the area of robotics and augmented realities due to the nature of the images to be processed in the systems for making the robot movements more efficiently. The machine learning researchers in the past proposed feed forward and back propagation neural network for providing effective training for making the systems intelligent. However, the training time is more and the accuracy achieved by such systems is not sufficient to make efficient decisions in many real life applications including computer vision. In such a scenario, many extensions were proposed by various researchers to the existing machine learning algorithms.
These algorithms increase the accuracy and reduce the convergence time. Among these, deep learning is currently the area of interest for researchers in the areas of artificial intelligence and machine learning. Moreover, the deep learning algorithms make use of a large-scale and hierarchical neural network which provides more efficiency through strong connections. One of the main advantages of deep learning algorithm is that, such algorithms provide great predictability with high accuracy and hence surpasses the other traditional machine learning algorithms which are used in various applications including image processing and speech recognition, natural language processing and intelligent question answering systems, cross lingual systems and machine translation [3,4].
In the past, the AlexNet system was proposed and it used a convolutional neural network that comprised of 60 million parameters as attributes and hence was able to come first in the ImageNet Large Scale Visual Recognition Challenge by providing higher accuracy of classification and reduced error rate when it was compared with other machine learning algorithms. After that, many extended forms of neural networks including radial basis function based neural networks were introduced and hence they were able to handle complex situations leading to the use of deep learning algorithms in many areas of image processing. One among the image processing application namely face recognition systems used a dataset of faces and classified them using neural networks and provided an accuracy rate of 97.45% in the effective recognition of faces which are available in a database of Labelled Faces in the Wild and it worked well by the optimization of transfer learning algorithms [5].
In image processing systems, feature extraction provides large number of features. Even in the benchmark datasets, the number of features is so high in such a way that they increase the time for classification. Moreover, many features obtained through feature extraction either do not contribute in the classification process or they provide only a negligible amount of contribution for decision making but increase the time exponentially. In such a scenario, the most contributing and important features must be identified. Most of the feature selection algorithms perform feature reduction based on the entropy values. Such techniques filter the features from the whole set of features. On the other hand, the incremental feature selection techniques select a subset of features and analyze them first. They add additional features after performing classification with initial set of features and by analyzing and adding other important features in order to perform feature optimization. Many optimization techniques are available in the literature for feature reduction including genetic algorithms, rule optimization techniques and linear programming models. Among them, swarm intelligence techniques and nature inspired techniques provide more smart decisions through intelligent decision making [6].
Genetic Algorithm (GA) is one such optimization method that has been researched by many researchers in the areas of artificial intelligence, machine learning, intrusion detection systems and image processing areas. The GAs follow the survival of the fittest model by selecting two parents at a time and producing children. An activation function is used to find the fittest children which will be used as the parents for the next generation. It provides operators like crossover and mutation to generate more number of off springs. The GA process uses a heuristic approach to solve the optimization problem by reducing the search space. It can either work individually in the decision making process or it can work with fuzzy rules and machine learning algorithms to solve an optimization problem more effectively. It uses a number of generation techniques for generating individuals from the given population. Therefore, it is more suitable for performing feature selection as well as classification [7].
Fuzzy set theory was proposed by Zadeh (1965) [8] and it has been used more frequently in the development of intelligent decision making systems because of its simplicity through rules and its similarity to human reasoning based on qualitative reasoning. Many fuzzy machine learning algorithms were used in the literature for generating and applying rules from a dataset. Such systems used standard membership functions for making efficient decisions. Higher order logics like fuzzy logic and temporal logic have the capability of performing reasoning under uncertainty through the gradation of truth values. Temporal logic is useful to analyze the past data and is also useful to predict the future using current and past data. The combination of fuzzy logic and temporal logic help to derive inference rules which can perform explanation based reasoning and analogy based reasoning. In image processing applications, fuzzy temporal rules can be combined with genetic algorithms for performing optimization in the process of feature selection and thus reducing the classification time leading to increase in classification accuracy.
In this work, a neuro fuzzy classifier developed using convolutional neural networks with a bias function for error reduction and fuzzy rules for weight adjustments is also proposed. Finally the rules derived from training using fuzzy neural networks are used to classify the similar objects for providing effective recognition. The proposed fuzzy neural network uses sigmoidal function as activated function. For feature selection triangular membership function is used to form fuzzy rules. On the other hand during classification trapezoidal membership function is used in order to improve the classification accuracy. The simulation is carried out using matlab simulation tool and the results obtained from this work have been compared with other existing systems in this area. From the experiments conducted using a robotic application scenario in which the robots are programmed for pick and place tasks which are captured by a sensor and having the image capturing system with a camera, it is found that the proposed model provides more realistic views of real world 3 Dimensional objects.
The image based recognition system proposed in this work consists of sub components namely recognition of vision information based on image processing techniques [9]. For this purpose are relaed an input video is divided into a number of frames. Among all these frames the most relevant frames to the robot object recognition system proposed in this work. It considers the key frames and analyses them based on color, texture, movement and depth details based on the dimensions of the image. It performs feature identification using fuzzy neural network and extracts the relevant features by applying fuzzy rules. From the extracted the features, most relevant features are selected by applying fuzzy rules and the selected features are used as input feature vector to the classifier. The main advantage of the proposed system is the increase in classification accuracy.
The rest of this paper is organized as follow's section 2 provides a survey of related works in the areas of augmented reality, neural network, fuzzy systems and classification algorithms. Section 3 explains the algorithms proposed in this paper along with the model proposed in this work. Section 4 depicts the results obtained from this work and also provides suitable discussions on the result. Section 5 gives conclusions on this work and also provides some suggestions for feature enhancements.

RELATED WORKS
There are many works that have been developed by various researchers in the direction of feature selection, classification and augmented reality in the past [7,8,[10][11][12][13][14]. Among them, Mohammad et al (2014) developed a novel algorithm called evolutionary oriented incremental algorithm for selecting the useful features by using their new framework. Moreover, they have tested their framework on an ordinary genetic algorithm and also developed new methods. In addition, their work employed with two new operators for performing the addition operation and deletion operation that are used for changing the length of the methods randomly. The major advantage of their algorithm is that it increases the classification accuracy and reduces the classification time.
Sannasi et al (2016) [15] introduced an intelligent Conditional Random Field based Feature Selection algorithm to develop an efficient intrusion detection system. Moreover, they have used intelligent algorithms for generating rules that are helpful for making effective decisions over the KDD Cup dataset which is used for evaluating their intrusion detection system. They have conducted number of experiments and proved that their feature selection algorithm takes less time and less number of features that are used to enhance the classification accuracy. Ganapathy et al. (2013) [3] discussed in detail about the intelligent techniques for performing feature selection and classification by applying the selected and contributing features. Moreover, they have focused over the selection of contributed features that are useful for performing classification process effectively for intrusion detection system.
Ulrich neumann and Suya (1999) [16] demonstrated that the augmented reality applications which direct the scene annotation, the pose stabilization and the extendible tracking range that are used for tracking the natural features in un-prepared environments. Yuan et al (2006) [5] developed a new registration algorithm that is used for estimating the method robustly, tracking the feature and for the reconstruction method which is also used for tracking the natural features. Moreover, they have super imposed and the virtual objects over the arbitrary region by the user preferred. Nitin et al (2018) [17] developed a novel method that is to find the object edges from image. Moreover, they have modified the fuzzy membership function that used to concentrate the multi-focus fusion over the fuzzy edge detection. In addition, they have proved their method by conducting the experiments with the satellite and color images that perform the fuse edge detection and compared with other existing systems. Akash Bapat et al (2016) [18] developed a novel scheme which is used a rolling shutter camera that used for tracking the 6 DOF head pose and also used to  [19] developed a new model that is a simple and linear iterative clustering model to construct the super pixels with a novel scheme and it is also generated the super pixels by using clustering method based on the proximity and the color similarity from the image plane. Ioannis et al (2012) [20] proposed a new transmission system, design, control and also the performance of the tele-operation force feedback. They have achieved better performance than the other existing systems. Kanimozhi et al (2018) [21] proposed a new fuzzy logic based prediction model called Intelligent Risk Prediction System for predicting the Breast Cancer disease by using the fuzzy temporal rules. They have applied intelligent fuzzy rules for performing the feature selection and classification processes over the standard benchmark dataset called Breast Cancer dataset which is available in UCI Machine Learning dataset.
Selvi et al (2016) [22] proposed fuzzy logic and time constraint based method for performing effective routing process over the wireless sensor networks. They have achieved better performance in terms of network lifetime and efficiency over the communication process. This is due to the fact that the use of fuzzy logic and the intelligence rules. Sethukkarasi et al (2014) [23] introduced an intelligent neural network based fuzzy logic which incorporates the time constrains for representing the knowledge model which is used for mining the temporal patterns. They have proved that their prediction system is better in the direction of diabetic and heart disease identification and detection when it is compared with the existing systems. Jaisankar et al (2012) [24] developed an intelligent agent based system by applying the fuzzy rough set and the outlier detection techniques for detecting the intruders. They have achieved better performance in terms of intrusion detection accuracy and false alarm rate. Sairamesh et al (2015) [25] proposed a new prediction system that is used for predicting the user interests to provide the relevant information by using the relevant feedback and the re-ranking process. The authors achieved better prediction accuracy than the existing systems.
A new model called heuristic mathematical model that has been proposed by Tahriri et al (2015) [6] that model uses a technique for optimizing the robot arm movement time that minimizes make-span and also maximized the total number of units produced each day in robot cells. Their model uses the users keys in the station, unit section and robot into the simulation software after creating 3D files and also sending them to the virtual reality system. Moreover, the data is converted into a text file and also it transferred to the robot arm movement with the help of time optimization algorithm for determining the optimum task sequence. WeiJie Wang. Hua Gen Wan [26] described the characteristics and the essential services of augmented reality about the real world applications that are used augmented reality. They have given brief history about the virtual reality and the augmented reality and the work flow of the augmented reality is also explained. Moreover, they have explained the different stages of the various augmented reality applications such as Image Acquisition process, the Feature Extraction process, the Feature Matching process, the Geometric Verification process and the associated information retrieval process. In addition, they highlighted the essential needs and the essence of the augmented reality in their work. However, most of the works present in the literature are not able to provide sufficient classification accuracy due to the lack of better feature selection. Hence a new feature selection algorithm is proposed this paper to handle the accuracy of classification.

SYSTEM ARCHITECTURE
The overall architecture of the proposed classification system is shown in Figure 1. It consists of seven major components namely Augmented Reality Image Dataset, Camera Capture Image Dataset, User Interface Module, Object Recognition System, Decision Manager, Temporal Information Manager, Fuzzy Rule Base and Knowledge base. Augmented Reality Image Dataset: A standard bench mark data set called AR image dataset is used for evaluating the proposed system which contains millions of images. Camera Capture Image Dataset: It consists of 1000 images that are taken by camera that are collected. User Interface Module: It collects the necessary data from these two dataset and it also decides what data have to be collected for evaluating the proposed system. The collected data are transferred to the object recognition system for further processes over the image data and received the recommended result from decision manager. Object Recognition System: It consists of four sub-systems namely the feature extraction, feature selection and object recognition as an image pre-processing subsystem and the classification subsystem which contains an existing classification algorithm called Convolutional Neural Network (CNN). In the pre-processing subsystem, it performs the pre-processing task by using a newly proposed pre-processing algorithm called Intelligent Agent based Incremental Feature Selection Algorithm (IAIFSA) for effective pre-processing of 198 the image datasets. In addition, an existing classification algorithm called Convolutional Neural Network algorithm that is used in this paper for performing effective classification of the image dataset. In this classification algorithm, there is a genetic algorithm process which uses temporal constraints and temporal fuzzy rules. Figure 1. System architecture Temporal Information Manager: It is useful for classification in the object recognition process of this proposed system. It also provides the necessary details about time constraints in the proposed system. Fuzzy Rule Base: It contains the necessary fuzzy rules for decision making over the dataset using a parallel approach for applying genetic operations and to form fuzzy temporal rules. Decision Manager: This decision manager helps to take final decision over the image data which are transferred from object recognition system with the help of Temporal Information manager and fuzzy rule base.

PROPOSED WORK
This section explains in detail about the proposed feature selection algorithm called intelligent incremental feature selection algorithm for selecting the most contributed features that are used to classify the image datasets. In this proposed feature selection algorithm, it extracts the relevant feature from the input images, recognize the images more relevancy and finalize the features according to the feature extraction and recognition results. Next, an existing classification algorithm called Convolutional Neural Network is used for classifying the images with the help of fuzzy rules that are generated by the proposed system.

Feature selection
In this work, an Intelligent Agent based Optimal Feature Selection Algorithm (IAOFSA) has been proposed in order to improve the decision accuracy based on classification of the images obtained from the virtual reality environment. This proposed feature selection algorithm has been developed by combining the optimization rules based on temporal constraints with intelligent agents in order to select the features which can reduce the classification time through temporal analysis of the dataset. This algorithm uses time constrained analysis on feature from the image which can vary over time to form a video sequence. Therefore, at each time instants the information gain ratio values are computed using the frequencies of occurrence of the features in the image and the rate at which it changes with respect to time. Here, second and millisecond level temporal analysis of image features are carried out in order to provide the optimal prediction for robotic arm movement. The main advantage of the proposed feature selection algorithm is that it reduces the classification time and increases the accuracy of next movement prediction through the results of the classifiers. This algorithm finds the information gain for the segments s1, s2, s3, …, sn obtained from the image at times t1, t2, ..., tn on the image I. The equation used to find the information gain value of the image I is given in (1-3). (1) Here, I is the dataset obtained the collection of frames called V1, V2, …, Vn. Si's represent the segments obtained from the image I, FR is the set features obtained from the image segments and Info is used to represent the information gain values and IGR represents the information gain ratio obtained for the ith frame of the video V. In this model, a set of frames collected from a video are considered as the images and they are segmented using a distance based approach that divides the image into eight equal segments. The detailed steps of the proposed IAOFSA are as follows: Intelligent Agent based Optimal Feature Selection Algorithm. Input: A video sequence V Output: A set of 'm' features FS collected from a set of 'n' images formed by the frames of V.
Step 1: Read the contents of video V.
Step 2: Initialize the set of features selected as NULL by setting FS = { } Step 3: Initialize i =1 Step 4: Call the frame formation agent to divide the video into 'n' frames and to make segments of equal size on each frame named as S1, S2, …, Sn.
Step 5: Call the computation agent to compute the value of Info (I, t) from the images using the formulae for the time instances t1 and t2 respectively.
This algorithm considers the move in eight directions namely front, back, left, right and the four diagonal directions for effective movement of the robotic arm. The intelligent classification algorithm applies fuzzy rules and genetic based optimization techniques in the neural network in order to find the most optimal movement in the promising directions.

RESULTS AND DISCUSSION
The proposed model has been proposed and implemented for predicting the augmented reality images accurately and it is provided sufficient data. Here, we have modeled a new Robot and also have been collected 50 images by conducting various experiments. The collected images are considered our own dataset in this work as input dataset. Figure 2 shows experimental setup of the new modelled robot by conducting various experiments. Table 1 shows the number of features extracted using existing algorithm and the proposed algorithm. Table 2 shows five important features that are selected by the proposed Intelligent Agent based Optimal Feature Selection Algorithm (IAOFSA). From Table 2, it listed out the important features such as angle, time, shape, start position(x, y) and the end position (x, y). These all features are used to improve the classification accuracy when test the robot images. Figure 3 shows the classification accuracy of the proposed algorithm and the existing algorithms when uses all the features of the robot images. Here, we have considered 12 features for making decision over the classifiers such as SVM, DT, Naïve Bayes, Random Forest and the proposed NGFRC. From Figure  3, it can be seen that the performance of the proposed algorithm is performed well with the uses of all the 12 features of robot images when it is compared with the existing classification algorithms such as SVM, DT, Naïve Bayes, Random Forest and the proposed NGFRC.  Start position(x,y) 6 End position (x,y) Figure 3. Classification accuracy with full features Figure 4 shows the classification accuracy of the proposed algorithm and the existing algorithms when uses all the features of the robot images. Here, we have considered only 6 features that are selected by the proposed feature selection algorithm for making decision over the classifiers such as SVM, DT, Naïve Bayes, Random Forest and the proposed NGFRC. From Figure 4, it can be seen that the performance of the proposed algorithm performs well with the use of the selected features from robot images when it is compared with the existing classification algorithms such as SVM, DT, Naïve Bayes, Random Forest and the proposed NGFRC.  Figure 5 shows the comparative analysis according to the classification accuracy of the proposed algorithm by conducting various experiments such as E1, E2, E3, E4 and E5. Here, we have considered full features [15] and the selected [5] features for performing classification process. From Figure 5, it can be observed that the performance of the proposed algorithm in all the experiments perform better well when it uses the selected features than when it uses the full features of the robot images that are uses in the experiments. This is due to the use of effective and more contributed features while making decision in the classification process. Moreover, the proposed classifier uses the neuro-genetic algorithm and fuzzy rules for the decision making process in the proposed classifier.

CONCLUSION AND FUTURE WORKS
In this paper, a new approach for implementing an Augmented Reality system by applying fuzzy genetic neural networks has been proposed and implemented for effective image classification and predicting the arm movement. It consists of two components namely feature selection and classification modules. The proposed model uses a fuzzy logic based incremental feature selection for extracting the relevant features that are used to recognize the important features from 3D images. Moreover, this paper explains the implementation and results of the proposed algorithms for an Augmented Reality system using image recognition, feature extraction, feature selection and classification by considering the global and local features of the images. For this purpose, we propose a three layer fuzzy neural network that has been implemented based on weight adjustments using fuzzy rules in the convolutional neural networks with genetic algorithm for effective optimization of rules. The classification algorithm is also based on fuzzy neuro-genetic approach which consists of two phases namely Training phase and testing phase. During the training phase, rules are formed based on objects and these rules are applied during the testing phase for recognizing the objects which can be used in robotics for effective object recognition. From the experiments conducted in this work, it is proved that the proposed model is more accurate in 3D object recognition.