Investigating echo state networks dynamics by means of recurrence analysis
Additional information
Authors
Bianchi F. M.,
Livi L.,
Alippi C.
Type
Journal Article
Year
2018
Language
English
Abstract
In this paper, we elaborate over the well-knowninterpretability issue in echo state networks. The idea is to investigatethe dynamics of reservoir neurons with time-series analysistechniques taken from research on complex systems. Notably, weanalyze time-series of neuron activations with Recurrence Plots(RPs) and Recurrence Quantification Analysis (RQA), whichpermit to visualize and characterize high-dimensional dynamicalsystems. We show that this approach is useful in a number ofways. First, the two-dimensional representation offered by RPsprovides a way for visualizing the high-dimensional dynamics ofa reservoir. Our results suggest that, if the network is stable,reservoir and input denote similar line patterns in the respectiveRPs. Conversely, the more unstable the ESN, the more the RPof the reservoir presents instability patterns. As a second result,we show that the Lmax measure is highly correlated with thewell-established maximal local Lyapunov exponent. This suggeststhat complexity measures based on RP diagonal lines distributionprovide a valuable tool to quantify the degree of network stability.Finally, our analysis shows that all RQA measures fluctuate onthe proximity of the so-called edge of stability, where an ESNtypically achieves maximum computational capability. We verifythat the determination of the edge of stability provided by suchRQA measures is more accurate than two well-known criteriabased on the Jacobian matrix of the reservoir. Therefore, weclaim that RPs and RQA-based analyses can be used as valuabletools to design an effective network given a specific problem.
Journal
IEEE Transactions on Neural Networks and Learning Systems
Volume
29
Month
February
Start page number
427
End page number
439
