Part I Introduction
1 Computational Intelligence: An Introduction................................................3
1.1 Introduction..............................................................................................3
1.2 Soft Computing.........................................................................................3
1.3 Probabilistic Reasoning............................................................................4
1.4 Evolutionary Computation........................................................................6
1.5 Computational Intelligence.......................................................................8
1.6 Hybrid Computational Technology..........................................................9
1.7 Application Areas...................................................................................10
1.8 Applications in Industry.........................................................................11
References..............................................................................................12
2 Traditional Problem Definition.....................................................................17
2.1 Introduction to Time Series Analysis.....................................................17
2.2 Traditional Problem Definition...............................................................18
2.2.1 Characteristic Features..............................................................18
2.2.1.1 Stationarity ..................................................................18
2.2.1.2 Linearity ......................................................................20
2.2.1.3 Trend............................................................................20
2.2.1.4 Seasonality...................................................................21
2.2.1.5 Estimation and Elimination of Trend and Seasonality...................................................................21
2.3 Classification of Time Series..................................................................22
2.3.1 Linear Time Series....................................................................23
2.3.2 Nonlinear Time Series...............................................................23
2.3.3 Univariate Time Series..............................................................23
2.3.4 Multivariate Time Series...........................................................24
2.3.5 Chaotic Time Series..................................................................24
2.4 Time Series Analysis..............................................................................25
2.4.1 Objectives of Analysis..............................................................25
2.4.2 Time Series Modelling..............................................................26
2.4.3 Time Series Models...................................................................26
2.5 Regressive Models..................................................................................27
2.5.1 Auto regression Model ..............................................................27
2.5.2 Moving-average Model ............................................................28
2.5.3 ARMA Model...........................................................................28
2.5.4 ARIMA Model..........................................................................29
2.5.5 CARMAX Model......................................................................32
2.5.6 Multivariate Time Series Model................................................33
2.5.7 Linear Time Series Models.......................................................35
2.5.8 Nonlinear Time Series Models..................................................35
2.5.9 Chaotic Time Series Models.....................................................36
2.6 Time-domain Models..............................................................................37
2.6.1 Transfer-function Models..........................................................37
2.6.2 State-space Models....................................................................38
2.7 Frequency-domain Models.....................................................................39
2.8 Model Building.......................................................................................42
2.8.1 Model Identification..................................................................43
2.8.2 Model Estimation......................................................................45
2.8.3 Model Validation and Diagnostic Check..................................48
2.9 Forecasting Methods...............................................................................49
2.9.1 Some Forecasting Issues...........................................................50
2.9.2 Forecasting Using Trend Analysis............................................51
2.9.3 Forecasting Using Regression Approaches...............................51
2.9.4 Forecasting Using the Box-Jenkins Method..............................53
2.9.4.1 Forecasting Using an Autoregressive Model AR(p)....53
2.9.4.2 Forecasting Using a Moving-average Model MA(q)...54
2.9.4.3 Forecasting Using an ARMA Model...........................54
2.9.4.4 Forecasting Using an ARIMA Model..........................56
2.9.4.5 Forecasting Using an CARIMAX Model....................57
2.9.5 Forecasting Using Smoothing...................................................57
2.9.5.1 Forecasting Using a Simple Moving Average.............57
2.9.5.2 Forecasting Using Exponential Smoothing .................58
2.9.5.3 Forecasting Using Adaptive Smoothing......................62
2.9.5.4 Combined Forecast......................................................64
2.10 Application Examples.............................................................................66
2.10.1 Forecasting Nonstationary Processes........................................66
2.10.2 Quality Prediction of Crude Oil................................................67
2.10.3 Production Monitoring and Failure Diagnosis..........................68
2.10.4 Tool Wear Monitoring..............................................................68
2.10.5 Minimum Variance Control......................................................69
2.10.6 General Predictive Control........................................................71
References..............................................................................................74
Selected Reading....................................................................................74
Part II Basic Intelligent Computational Technologies
3 Neural Networks Approach...........................................................................79
3.1 Introduction............................................................................................79
3.2 Basic Network Architecture....................................................................80
3.3 Networks Used for Forecasting..............................................................84
3.3.1 Multilayer Perceptron Networks...............................................84
3.3.2 Radial Basis Function Networks...............................................85
3.3.3 Recurrent Networks ..................................................................87
3.3.4 Counter Propagation Networks.................................................92
3.3.5 Probabilistic Neural Networks..................................................94
3.4 Network Training Methods.....................................................................95
3.4.1 Accelerated Backpropagation Algorithm..................................99
3.5 Forecasting Methodology.....................................................................103
3.5.1 Data Preparation for Forecasting.............................................104
3.5.2 Determination of Network Architecture..................................106
3.5.3 Network Training Strategy......................................................112
3.5.4 Training, Stopping and Evaluation..........................................116
3.6 Forecasting Using Neural Networks.....................................................129
3.6.1 Neural Networks versus Traditional Forecasting....................129
3.6.2 Combining Neural Networks and Traditional Approaches.....131
3.6.3 Nonlinear Combination of Forecasts Using Neural Networks 132
3.6.4 Forecasting of Multivariate Time Series.................................136 References............................................................................................137
Selected Reading..................................................................................142
4 Fuzzy Logic Approach .................................................................................143
4.1 Introduction..........................................................................................143
4.2 Fuzzy Sets and Membership Functions................................................144
4.3 Fuzzy Logic Systems ...........................................................................146
4.3.1 Mamdani Type of Fuzzy Logic Systems.................................148
4.3.2 Takagi-Sugeno Type of Fuzzy Logic Systems........................148
4.3.3 Relational Fuzzy Logic System of Pedrycz.............................149
4.4 Inferencing the Fuzzy Logic System....................................................150
4.4.1 Inferencing a Mamdani-type Fuzzy Model.............................150
4.4.2 Inferencing a Takagi-Sugeno-type Fuzzy Model....................153
4.4.3 Inferencing a (Pedrycz) Relational Fuzzy Model....................154
4.5 Automated Generation of Fuzzy Rule Base..........................................157
4.5.1 The Rules Generation Algorithm............................................157
4.5.2 Modifications Proposed for Automated Rules Generation......162
4.5.3 Estimation of Takagi-Sugeno Rules’ Consequent Parameters...............................................................................166
4.6 Forecasting Time Series Using the Fuzzy Logic Approach..................169
4.6.1 Forecasting Chaotic Time Series: An Example.......................169
4.7 Rules Generation by Clustering............................................................173
4.7.1 Fuzzy Clustering Algorithms for Rule Generation..................173
4.7.1.1 Elements of Clustering Theory .................................174
4.7.1.2 Hard Partition............................................................175
4.7.1.3 Fuzzy Partition...........................................................177
4.7.2 Fuzzy c-means Clustering.......................................................178
4.7.2.1 Fuzzy c-means Algorithm..........................................179
4.7.2.1.1 Parameters of Fuzzy c-means Algorithm....180
4.7.3 Gustafson-Kessel Algorithm...................................................183
4.7.3.1 Gustafson-Kessel Clustering Algorithm....................184
4.7.3.1.1 Parameters of Gustafson-Kessel Algorithm....................................................185
4.7.3.1.2 Interpretation of Cluster Covariance Matrix.........................................................185
4.7.4 Identification of Antecedent Parameters by Fuzzy Clustering................................................................................185
4.7.5 Modelling of a Nonlinear Plant...............................................187
4.8 Fuzzy Model as Nonlinear Forecasts Combiner...................................190
4.9 Concluding Remarks............................................................................193
References............................................................................................193
5 Evolutionary Computation..........................................................................195
5.1 Introduction..........................................................................................195
5.1.1 The Mechanisms of Evolution................................................196
5.1.2 Evolutionary Algorithms.........................................................196
5.2 Genetic Algorithms...............................................................................197
5.2.1 Genetic Operators....................................................................198
5.2.1.1 Selection....................................................................199
5.2.1.2 Reproduction.............................................................199
5.2.1.3 Mutation ....................................................................199
5.2.1.4 Crossover...................................................................201
5.2.2 Auxiliary Genetic Operators...................................................201
5.2.2.1 Fitness Windowing or Scaling...................................201
5.2.3 Real-coded Genetic Algorithms..............................................203
5.2.3.1 Real Genetic Operators..............................................204
5.2.3.1.1 Selection Function......................................204
5.2.3.1.2 Crossover Operators for Real-coded Genetic Algorithms.....................................205
5.2.3.1.3 Mutation Operators.....................................205
5.2.4 Forecasting Examples.............................................................206
5.3 Genetic Programming...........................................................................209
5.3.1 Initialization............................................................................210
5.3.2 Execution of Algorithm...........................................................211
5.3.3 Fitness Measure.......................................................................211
5.3.4 Improved Genetic Versions.....................................................211
5.3.5 Applications............................................................................212
5.4 Evolutionary Strategies.........................................................................212
5.4.1 Applications to Real-world Problems ....................................213
5.5 Evolutionary Programming ..................................................................214
5.5.1 Evolutionary Programming Mechanism ................................215
5.6 Differential Evolution ..........................................................................215
5.6.1 First Variant of Differential Evolution (DE1).........................216
5.6.2 Second Variant of Differential Evolution (DE2).....................218 References............................................................................................218
Part III Hybrid Computational Technologies
6 Neuro-fuzzy Approach.................................................................................223
6.1 Motivation for Technology Merging....................................................223
6.2 Neuro-fuzzy Modelling ........................................................................224
6.2.1 Fuzzy Neurons........................................................................227
6.2.1.1 AND Fuzzy Neuron...................................................228
6.2.1.2 OR Fuzzy Neuron......................................................229
6.3 Neuro-fuzzy System Selection for Forecasting....................................230
6.4 Takagi-Sugeno-type Neuro-fuzzy Network..........................................232
6.4.1 Neural Network Representation of Fuzzy Logic Systems.......233
6.4.2 Training Algorithm for Neuro-fuzzy Network........................234
6.4.2.1 Backpropagation Training of Takagi-Sugeno-type Neuro-fuzzy Network................................................234
6.4.2.2 Improved Backpropagation Training Algorithm.......238
6.4.2.3 Levenberg-Marquardt Training Algorithm................239
6.4.2.3.1 Computation of Jacobian Matrix ...............241
6.4.2.4 Adaptive Learning Rate and Oscillation Control ......246
6.5 Comparison of Radial Basis Function Network and Neuro-fuzzy Network ..........................................................................247
6.6 Comparison of Neural Network and Neuro-fuzzy Network Training..248
6.7 Modelling and Identification of Nonlinear Dynamics .........................249
6.7.1 Short-term Forecasting of Electrical load ...............................249
6.7.2 Prediction of Chaotic Time Series...........................................253
6.7.3 Modelling and Prediction of Wang Data.................................258
6.8 Other Engineering Application Examples............................................264
6.8.1 Application of Neuro-fuzzy Modelling to Materials Property Prediction .................................................265
6.8.1.1 Property Prediction for C-Mn Steels ..........................266
6.8.1.2 Property Prediction for C-Mn-Nb Steels ....................266
6.8.2 Correction of Pyrometer Reading ...........................................266
6.8.3 Application for Tool Wear Monitoring ..................................268
6.9 Concluding Remarks............................................................................270 References............................................................................................271
7 Transparent Fuzzy/Neuro-fuzzy Modelling ..............................................275
7.1 Introduction .........................................................................................275
7.2 Model Transparency and Compactness................................................276
7.3 Fuzzy Modelling with Enhanced Transparency....................................277
7.3.1 Redundancy in Numerical Data-driven Modelling .................277
7.3.2 Compact and Transparent Modelling Scheme ........................279
7.4 Similarity Between Fuzzy Sets.............................................................281
7.4.1 Similarity Measure..................................................................282
7.4.2 Similarity-based Rule Base Simplification .............................282
7.5 Simplification of Rule Base..................................................................285
7.5.1 Merging Similar Fuzzy Sets....................................................287
7.5.2 Removing Irrelevant Fuzzy Sets.............................................289
7.5.3 Removing Redundant Inputs...................................................290
7.5.4 Merging Rules ........................................................................290
7.6 Rule Base Simplification Algorithms ..................................................291
7.6.1 Iterative Merging.....................................................................292
7.6.2 Similarity Relations.................................................................294
7.7 Model Competitive Issues: Accuracy versus Complexity....................296
7.8 Application Examples...........................................................................299
7.9 Concluding Remarks............................................................................302 References............................................................................................302
8 Evolving Neural and Fuzzy Systems...........................................................305
8.1 Introduction..........................................................................................305
8.1.1 Evolving Neural Networks......................................................305
8.1.1.1 Evolving Connection Weights...................................306
8.1.1.2 Evolving the Network Architecture...........................309
8.1.1.3 Evolving the Pure Network Architecture...................310
8.1.1.4 Evolving Complete Network.....................................311
8.1.1.5 Evolving the Activation Function..............................312
8.1.1.6 Application Examples................................................313
8.1.2 Evolving Fuzzy Logic Systems...............................................313 References............................................................................................317
9 Adaptive Genetic Algorithms.......................................................................321
9.1 Introduction..........................................................................................321
9.2 Genetic Algorithm Parameters to Be Adapted......................................322
9.3 Probabilistic Control of Genetic Algorithm Parameters.......................323
9.4 Adaptation of Population Size..............................................................327
9.5 Fuzzy-logic-controlled Genetic Algorithms.........................................329
9.6 Concluding Remarks............................................................................330 References............................................................................................330
Part IV Recent Developments
10 State of the Art and Development Trends..................................................335
10.1 Introduction..........................................................................................335
10.2 Support Vector Machines.....................................................................337
10.2.1 Data-dependent Representation...............................................342
10.2.2 Machine Implementation.........................................................343
10.2.3 Applications............................................................................344
10.3 Wavelet Networks................................................................................345 10.3.1 Wavelet Theory.......................................................................345
10.3.2 Wavelet Neural Networks.......................................................346
10.3.3 Applications............................................................................349
10.4 Fractally Configured Neural Networks.................................................350
10.5 Fuzzy Clustering...................................................................................352 10.5.1 Fuzzy Clustering Using Kohonen Networks...........................353
10.5.2 Entropy-based Fuzzy Clustering.............................................355
10.5.2.1 Entropy Measure for Cluster Estimation...................356
10.5.2.1 The Entropy Measure ..................................356
10.5.2.2 Fuzzy Clustering Based on Entropy Measure............358
10.5.2.3 Fuzzy Model Identification Using Entropy-based Fuzzy Clustering................................359 References............................................................................................360
Index....................................................................................................................363
· · · · · · (收起)