-*- mode: org -*-
Miscellaneous applications and hybrids (only a few papers on each...)
*  32(<-120): A niching genetic programming-based multi-objective algorithm for hybrid data classification

This paper introduces a multi-objective algorithm based on genetic
programming to extract classification rules in databases composed of
hybrid data, i.e., regular (e.g. numerical, logical, and textual) and
non-regular (e.g. geographical) attributes. This algorithm employs a
niche technique combined with a population archive in order to
identify the rules that are more suitable for classifying items
amongst classes of a given data set. The algorithm is implemented in
such a way that the user can choose the function set that is more
adequate for a given application. This feature makes the proposed
approach virtually applicable to any kind of data set classification
problem. Besides, the classification problem is modeled as a
multi-objective one, in which the maximization of the accuracy and the
minimization of the classifier complexity are considered as the
objective functions. A set of different classification problems, with
considerably different data sets and domains, has been considered:
wines, patients with hepatitis, incipient faults in power transformers
and level of development of cities. In this last data set, some of the
attributes are geographical, and they are expressed as points, lines
or polygons. The effectiveness of the algorithm has been compared with
three other methods, widely employed for classification: Decision Tree
(C4.5), Support Vector Machine (SVM) and Radial Basis Function
(RBF). Statistical comparisons have been conducted employing one-way
ANOVA and Tukey's tests, in order to provide reliable comparison of
the methods. The results show that the proposed algorithm achieved
better classification effectiveness in all tested instances, what
suggests that it is suitable for a considerable range of
classification applications. (C) 2014 Elsevier B.V. All rights
reserved.

2014



*  40(<-467): AG-ART: An adaptive approach to evolving ART architectures

This paper focuses on classification problems, and in particular on
the evolution of ARTMAP architectures using genetic algorithms, with
the objective of improving generalization performance and alleviating
the adaptive resonance theory (ART) category proliferation problem. In
a previous effort, we introduced evolutionary fuzzy ARTMAP (FAM),
referred to as genetic Fuzzy ARTMAP (GFAM). In this paper we apply an
improved genetic algorithm to FAM and extend these ideas to two other
ART architectures; ellipsoidal ARTMAP (EAM) and Gaussian ARTMAP
(CAM). One of the major advantages of the proposed improved genetic
algorithm is that it adapts the CA parameters automatically, and in a
way that takes into consideration the intricacies of the
classification problem under consideration. The resulting genetically
engineered ART architectures are justifiably referred to as AG-FAM,
AG-EAM and AG-GAM or collectively as AG-ART (adaptive genetically
engineered ART). We compare the performance (in terms of accuracy,
size, and computational cost) of the AG-ART architectures with GFAM,
and other ART architectures that have appeared in the literature and
attempted to solve the category proliferation problem. Our results
demonstrate that AG-ART architectures exhibit better performance than
their other ART counterparts (semi-supervised ART) and better
performance than GFAM. We also compare AG-ART's performance to other
related results published in the classification literature, and
demonstrate that AG-ART architectures exhibit competitive
generalization performance and, quite often, produce smaller size
classifiers in solving the same classification problems. We also show
that AG-ART's performance gains are achieved within a reasonable
computational budget. (C) 2008 Elsevier B.V. All rights reserved.

2009




* 270(<-418): An Adaptive Multiobjective Approach to Evolving ART Architectures

In this paper, we present the evolution of adaptive resonance theory
(ART) neural network architectures (classifiers) using a
multiobjective optimization approach. In particular, we propose the
use of a multiobjective evolutionary approach to simultaneously evolve
the weights and the topology of three well-known ART architectures;
fuzzy ARTMAP (FAM), ellipsoidal ARTMAP (EAM), and Gaussian ARTMAP
(GAM). We refer to the resulting architectures as MO-GFAM, MO-GEAM,
and MO-GGAM, and collectively as MO-GART. The major advantage of
MO-GART is that it produces a number of solutions for the
classification problem at hand that have different levels of merit
[accuracy on unseen data (generalization) and size (number of
categories created)]. MO-GART is shown to be more elegant (does not
require user intervention to define the network parameters), more
effective (of better accuracy and smaller size), and more efficient
(faster to produce the solution networks) than other ART neural
network architectures that have appeared in the
literature. Furthermore, MO-GART is shown to be competitive with other
popular classifiers, such as classification and regression tree (CART)
and support vector machines (SVMs).

2010





*  43(<-238): Multiplicative Update Rules for Concurrent Nonnegative Matrix Factorization and Maximum Margin Classification

The state-of-the-art classification methods which employ nonnegative
matrix factorization (NMF) employ two consecutive independent
steps. The first one performs data transformation (dimensionality
reduction) and the second one classifies the transformed data using
classification methods, such as nearest neighbor/centroid or support
vector machines (SVMs). In the following, we focus on using NMF
factorization followed by SVM classification. Typically, the
parameters of these two steps, e. g., the NMF bases/coefficients and
the support vectors, are optimized independently, thus leading to
suboptimal classification performance. In this paper, we merge these
two steps into one by incorporating maximum margin classification
constraints into the standard NMF optimization. The notion behind the
proposed framework is to perform NMF, while ensuring that the margin
between the projected data of the two classes is maximal. The
concurrent NMF factorization and support vector optimization are
performed through a set of multiplicative update rules. In the same
context, the maximum margin classification constraints are imposed on
the NMF problem with additional discriminant constraints and
respective multiplicative update rules are extracted. The impact of
the maximum margin classification constraints on the NMF factorization
problem is addressed in Section VI. Experimental results in several
databases indicate that the incorporation of the maximum margin
classification constraints into the NMF and discriminant NMF objective
functions improves the accuracy of the classification.

2013





*  48(<- 26): Joint model for feature selection and parameter optimization coupled with classifier ensemble in chemical mention recognition

Mention recognition in chemical texts plays an important role in a
wide-spread range of application areas. Feature selection and
parameter optimization are the two important issues in machine
learning. While the former improves the quality of a classifier by
removing the redundant and irrelevant features, the later concerns
finding the most suitable parameter values, which have significant
impact on the overall classification performance. In this paper we
formulate a joint model that performs feature selection and parameter
optimization simultaneously, and propose two approaches based on the
concepts of single and multiobjective optimization
techniques. Classifier ensemble techniques are also employed to
improve the performance further. We identify and implement variety of
features that are mostly domain-independent. Experiments are performed
with various configurations on the benchmark patent and Medline
datasets. Evaluation shows encouraging performance in all the
settings. (C) 2015 Elsevier B.V. All rights reserved.

2015





*  53(<- 44): The influence of scaling metabolomics data on model classification accuracy

Correctly measured classification accuracy is an important aspect not
only to classify pre-designated classes such as disease versus control
properly, but also to ensure that the biological question can be
answered competently. We recognised that there has been minimal
investigation of pre-treatment methods and its influence on
classification accuracy within the metabolomics literature. The
standard approach to pre-treatment prior to classification modelling
often incorporates the use of methods such as autoscaling, which
positions all variables on a comparable scale thus allowing one to
achieve separation of two or more groups (target classes). This is
often undertaken without any prior investigation into the influence of
the pre-treatment method on the data and supervised learning
techniques employed. Whilst this is useful for deriving essential
information such as predictive ability or visual interpretation in
many cases, as shown in this study the standard approach is not always
the most suitable option available. Here, a study has been conducted
to investigate the influence of six pre-treatment methods-autoscaling,
range, level, Pareto and vast scaling, as well as no scaling-on four
classification models, including: principal components-discriminant
function analysis (PC-DFA), support vector machines (SVM), random
forests (RF) and k-nearest neighbours (kNN)-using three publically
available metabolomics data sets. We have demonstrated that
undertaking different pre-treatment methods can greatly affect the
interpretation of the statistical modelling outputs. The results have
shown that data pre-treatment is context dependent and that there was
no single superior method for all the data sets used. Whilst we did
find that vast scaling produced the most robust models in terms of
classification rate for PC-DFA of both NMR spectroscopy data sets, in
general we conclude that both vast scaling and autoscaling produced
similar and superior results in comparison to the other four
pre-treatment methods on both NMR and GC-MS data sets. It is therefore
our recommendation that vast scaling is the primary pre-treatment
method to use as this method appears to be more stable and robust
across all the different classifiers that were conducted in this
study.

2015



*  69(<-369): Classification as Clustering: A Pareto Cooperative-Competitive GP Approach

Intuitively population based algorithms such as genetic programming
provide a natural environment for supporting solutions that learn to
decompose the overall task between multiple individuals, or a
team. This work presents a framework for evolving teams without
recourse to prespecifying the number of cooperating individuals. To do
so, each individual evolves a mapping to a distribution of outcomes
that, following clustering, establishes the parameterization of a
(Gaussian) local membership function. This gives individuals the
opportunity to represent subsets of tasks, where the overall task is
that of classification under the supervised learning domain. Thus,
rather than each team member representing an entire class, individuals
are free to identify unique subsets of the overall classification
task. The framework is supported by techniques from evolutionary
multiobjective optimization (EMO) and Pareto competitive
coevolution. EMO establishes the basis for encouraging individuals to
provide accurate yet nonoverlaping behaviors; whereas competitive
coevolution provides the mechanism for scaling to potentially large
unbalanced datasets. Benchmarking is performed against recent examples
of nonlinear SVM classifiers over 12 UCI datasets with between 150 and
200,000 training instances. Solutions from the proposed coevolutionary
multiobjective GP framework appear to provide a good balance between
classification performance and model complexity, especially as the
dataset instance count increases.

2011




*  82(<-330): Integrating multicriteria PROMETHEE II method into a single-layer perceptron for two-class pattern classification

PROMETHEE methods based on the outranking relation theory are
extensively used in multicriteria decision aid. A preference index
representing the intensity of preference for one pattern over another
pattern can be measured by various preference functions. The higher
the intensity, the stronger the preference is indicated. In contrast
to traditional single-layer perceptrons (SLPs) with the sigmoid
function, this paper develops a novel PROMETHEE II-based SLP using
concepts from the PROMETHEE II method involving pairwise comparisons
between patterns. The assignment of a class label to a pattern is
dependent on its net preference index, which the proposed perceptron
obtains. Specially, this study designs a genetic-algorithm-based
learning algorithm to determine the relative weights of respective
criteria in order to derive the preference index for any pair of
patterns. Computer simulations involving several real-world data sets
reveal the classification performance of the proposed PROMETHEE
II-based SLP. The proposed perceptron performs well compared to the
other well-known fuzzy or non-fuzzy classification methods.

2011



*  84(<-399): A single-layer perceptron with PROMETHEE methods using novel preference indices

The Preference Ranking Organization METHods for Enrichment Evaluations
(PROMETHEE) methods, based on the outranking relation theory, are used
extensively in multi-criteria decision aid (MCDA). In particular,
preference indices with weighted average aggregation representing the
intensity of preference for one pattern over another pattern are
measured by various preference functions. The higher the intensity,
the stronger the preference is indicated. For MCDA, to obtain the
ranking of alternatives, compromise operators such as the weighted
average aggregation, or the disjunctive operators are often employed
to aggregate the performance values of criteria. The compromise
operators express the group utility or the majority rule, whereas the
disjunctive operators take into account the strongly opponent or
agreeable minorities. Since these two types of operators have their
own unique features, it is interesting to develop a novel aggregator
by integrating them into a single aggregator for a preference
index. This study aims to develop a novel PROMETHEE-based single-layer
perceptron (PROSLP) for pattern classification using the proposed
preference index. The assignment of a class label to a pattern is
dependent on its net preference index, which is obtained by the
proposed perceptron. Computer simulations involving several real-world
data sets reveal the classification performance of the proposed
PROMETHEE-based SLP. The proposed perceptron with the novel preference
index performs well compared to that with the original one. (C) 2010
Elsevier B.V. All rights reserved.

2010






* 227(<-505): Learning based brain emotional intelligence as a new aspect for development of an alarm system

The multi criteria and purposeful prediction approach has been
introduced and is implemented by the fast and efficient behavioral
based brain emotional learning method. On the other side, the
emotional learning from brain model has shown good performance and is
characterized by high generalization property. New approach is
developed to deal with low computational and memory resources and can
be used with the largest available data sets. The scope of paper is to
reveal the advantages of emotional learning interpretations of brain
as a purposeful forecasting system designed to warning; and to make a
fair comparison between the successful neural (MLP) and neurofuzzy
(ANFIS) approaches in their best structures and according to
prediction accuracy, generalization, and computational complexity. The
auroral electrojet (AE) index are used as practical examples of
chaotic time series and introduced method used to make predictions and
warning of geomagnetic disturbances and geomagnetic storms based on AE
index.

2008



* 304(<-632): Multicriteria fuzzy classification procedure PROCFTN: methodology and medical application

In this paper, we introduce a new classification procedure for
assigning objects to predefined classes, named PROCFTN. This procedure
is based on a fuzzy scoring function for choosing a subset of
prototypes, which represent the closest resemblance with an object to
be assigned. It then applies the majority-voting rule to assign an
object to a class. We also present a medical application of this
procedure as an aid to assist the diagnosis of central nervous system
tumours. The results are compared with those obtained by other
classification methods, reported on the same data set, including
decision tree, production rules, neural network, k nearest neighbor,
multilayer perceptron and logistic regression. Our results are very
encouraging and show that the multicriteria decision analysis approach
can be successfully used to help medical diagnosis. Crown Copyright
(C) 2003 Published by Elsevier B.V. All rights reserved.

2004



* 316(<- 52): Multiple Actor-Critic Structures for Continuous-Time Optimal Control Using Input-Output Data

In industrial process control, there may be multiple performance
objectives, depending on salient features of the input-output
data. Aiming at this situation, this paper proposes multiple
actor-critic structures to obtain the optimal control via input-output
data for unknown nonlinear systems. The shunting inhibitory artificial
neural network (SIANN) is used to classify the input-output data into
one of several categories. Different performance measure functions may
be defined for disparate categories. The approximate dynamic
programming algorithm, which contains model module, critic network,
and action network, is used to establish the optimal control in each
category. A recurrent neural network (RNN) model is used to
reconstruct the unknown system dynamics using input-output data. NNs
are used to approximate the critic and action networks,
respectively. It is proven that the model error and the closed unknown
system are uniformly ultimately bounded. Simulation results
demonstrate the performance of the proposed optimal control scheme for
the unknown nonlinear system.

2015



* 321(<-610): Evolutionary multi-objective optimization for simultaneous generation of signal-type and symbol-type representations

It has been a controversial issue in the research of cognitive science
and artificial intelligence whether signal-type representations
(typically connectionist networks) or symbol-type representations
(e.g., semantic networks, production systems) should be
used. Meanwhile, it has also been recognized that both types of
information representations might exist in the human brain. In
addition, symbol-type representations are often very helpful in
gaining insights into unknown systems. For these reasons,
comprehensible symbolic rules need to be extracted from trained neural
networks. In this paper, an evolutionary multi-objective algorithm is
employed to generate multiple models that facilitate the generation of
signal-type and symbol-type representations simultaneously. It is
argued that one main difference between signal-type and symbol-type
representations lies in the fact that the signal-type representations,
are models of a higher complexity (fine representation), whereas
symbol-type representations are models of a lower complexity (coarse
representation). Thus, by generating models with a spectrum of model
complexity, we are able to obtain a population of models of both
signal-type and symbol-type quality, although certain post-processing
is needed to get a fully symbol-type representation. An illustrative
example is given on generating neural networks for the breast cancer
diagnosis benchmark problem.

2005



* 343(<-675): Pattern classification by linear goal programming and its extensions

Pattern classification is one of the main themes in pattern
recognition, and has been tackled by several methods such as the
statistic one, artificial neural networks, mathematical programming
and so on. Among them, the multi-surface method proposed by
Mangasarian is very attractive, because it can provide an exact
discrimination function even for highly nonlinear problems without any
assumption on the data distribution. However, the method often causes
many slits on the discrimination curve. In other words, the piecewise
linear discrimination curve is sometimes too complex resulting in a
poor generalization ability. In this paper, several trials in order to
overcome the difficulties of the multi-surface method are
suggested. One of them is the utilization of goal programming in which
the auxiliary linear programming problem is formulated as a goal
programming in order to get as simple discrimination curves as
possible. Another one is to apply fuzzy programming by which we can
get fuzzy discrimination curves with gray zones. In addition, it will
be shown that using the suggested methods, the additional learning can
be easily made. These features of the methods make the discrimination
more realistic. The effectiveness of the methods is shown on the basis
of some applications.

1998



* 379(<-274): Computational algorithms inspired by biological processes and evolution

In recent times computational algorithms inspired by biological
processes and evolution are gaining much popularity for solving
science and engineering problems. These algorithms are broadly
classified into evolutionary computation and swarm intelligence
algorithms, which are derived based on the analogy of natural
evolution and biological activities. These include genetic algorithms,
genetic programming, differential evolution, particle swarm
optimization, ant colony optimization, artificial neural networks,
etc. The algorithms being random-search techniques, use some
heuristics to guide the search towards optimal solution and speed-up
the convergence to obtain the global optimal solutions. The
bio-inspired methods have several attractive features and advantages
compared to conventional optimization solvers. They also facilitate
the advantage of simulation and optimization environment
simultaneously to solve hard-to-define (in simple expressions),
real-world problems. These biologically inspired methods have provided
novel ways of problem-solving for practical problems in traffic
routing, networking, games, industry, robotics, economics, mechanical,
chemical, electrical, civil, water resources and others fields. This
article discusses the key features and development of bio-inspired
computational algorithms, and their scope for application in science
and engineering fields.

2012




* 386(<-683): FUZZY THRESHOLD FUNCTIONS AND APPLICATIONS

The set of fuzzy threshold functions is defined to be a fuzzy set over
the set of functions. All threshold functions have full memberships in
this fuzzy set. Defines and investigates a distance measure between a
non-linearly separable function and the set of all threshold
functions. Defines an explicit expression for the membership function
of a fuzzy threshold function through the use of this distance measure
and finds three upper bounds for this measure. Presents a general
method to compute the distance, an algorithm to generate the
representation automatically, and a procedure to determine the proper
weights and thresholds automatically. Presents the relationships among
threshold gate networks, artificial neural networks and fuzzy neural
networks. The results may have useful applications in logic design,
pattern recognition, fuzzy logic, multi-objective fuzzy optimization
and related areas.

1995



* 395(<-215): A novel artificial immune clonal selection classification and rule mining with swarm learning model

Metaheuristic optimisation algorithms have become popular choice for
solving complex problems. By integrating Artificial Immune clonal
selection algorithm (CSA) and particle swarm optimisation (PSO)
algorithm, a novel hybrid Clonal Selection Classification and Rule
Mining with Swarm Learning Algorithm (CS2) is proposed. The main goal
of the approach is to exploit and explore the parallel computation
merit of Clonal Selection and the speed and self-organisation merits
of Particle Swarm by sharing information between clonal selection
population and particle swarm. Hence, we employed the advantages of
PSO to improve the mutation mechanism of the artificial immune CSA and
to mine classification rules within datasets. Consequently, our
proposed algorithm required less training time and memory cells in
comparison to other AIS algorithms. In this paper, classification rule
mining has been modelled as a miltiobjective optimisation problem with
predictive accuracy. The multiobjective approach is intended to allow
the PSO algorithm to return an approximation to the accuracy and
comprehensibility border, containing solutions that are spread across
the border. We compared our proposed algorithm classification accuracy
CS2 with five commonly used CSAs, namely: AIRS1, AIRS2, AIRS-Parallel,
CLONALG, and CSCA using eight benchmark datasets. We also compared our
proposed algorithm classification accuracy CS2 with other five
methods, namely: Naive Bayes, SVM, MLP, CART, and RFB. The results
show that the proposed algorithm is comparable to the 10 studied
algorithms. As a result, the hybridisation, built of CSA and PSO, can
develop respective merit, compensate opponent defect, and make
search-optimal effect and speed better.

2013