Skip to main content

Advertisement

Springer Nature Link
Log in

Complex Valued Deep Neural Networks for Nonlinear System Modeling

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Deep learning models, such as convolutional neural networks (CNN), have been successfully applied in pattern recognition and system identification recent years. But for the cases of missing data and big noises, CNN does not work well for dynamic system modeling. In this paper, complex valued convolution neural network (CVCNN) is presented for modeling nonlinear systems with large uncertainties. Novel training methods are proposed for CVCNN. Comparisons with other classical neural networks are made to show the advantages of the proposed methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Noel JP, Kerschen G (2017) Nonlinear system identification in structural dynamics: 10 more years of progress. Mech Syst Signal Process 83:2–35

    Article  Google Scholar 

  2. Lopez M, Morales J, Yu W (2020) Frequency domain CNN and disipate energy approach for damage detection in building structures. Soft Comput 24:15821–1584051

    Article  Google Scholar 

  3. LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551

    Article  Google Scholar 

  4. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  5. Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. In: Proceedings of the advances in neural information processing systems (NIPS’06), pp. 153–160

  6. Hinton G, Osindero S, Teh Y (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  7. Sermanet P, LeCun Y (2011) Traffic sign recognition with multi-scale convolutional networks. In: Proceedings of the 2011 international joint conference on neural networks (IJCNN)

  8. Mao L, Li X, Yang D et al (2021) Convolutional feature frequency adaptive fusion object detection network. Neural Process Lett. https://doi.org/10.1007/s11063-021-10560-4

    Article  Google Scholar 

  9. Soleymanpour S, Sadr H, Beheshti H (2020) An efficient deep learning method for encrypted traffic classification on the web. In: 2020 6th International conference on web research (ICWR), IEEE, pp. 209–216

  10. Sultana F, Sufian A, Dutta P (2018). Advancements in image classification using convolutional neural network. In: 2018 Fourth international conference on research in computational intelligence and communication networks (ICRCICN), IEEE, pp. 122–129

  11. Yadav SS, Jadhav SM (2019) Deep convolutional neural network based medical image classification for disease diagnosis. J Big Data 6(1):1–18

    Article  Google Scholar 

  12. Altan G, Yayik A, Kutlu Y (2021) Deep learning with ConvNet predicts imagery tasks through EEG. Neural Process Lett 53:1–16

    Article  Google Scholar 

  13. Ismail FH et al (2019) Deep learning for time series classification: a review. Data Min Knowl Discov 33(4):917–963

    Article  MathSciNet  MATH  Google Scholar 

  14. Hatami N, Gavet Y, Debayle J (2017) Classification of time-series images using deep convolutional neural networks. arXiv:1710.00886v2

  15. Wang Y et al. (2016) CNNpack: Packing convolutional neural networks in the frequency domain. In: NIPS

  16. Lee H, Groose R, Ranganath R, Ng A (2009) Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: Proceedings of the 26th international conference on machine learning ( ICML09), pp. 609–616

  17. Kiranyaz S, Avci O, Abdeljaber O, Ince T, Gabbouj M, Inman DJ (2021) 1D convolutional neural networks and applications: a survey. Mech Syst Signal Process 151:107398

    Article  Google Scholar 

  18. Borovykh A, Bohte S, Oosterlee CW (2017) Conditional time series forecasting with convolutional neural networks. arXiv preprint arXiv:1703.04691

  19. Javedani SH et al (2019) Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series. Energy 175:365–377

    Article  Google Scholar 

  20. Binkowski M, Marti G, Donnat P (2017) Autoregressive convolutional neural networks for asynchronous time series. arXiv:1703.04122

  21. Namasudra S, Dhamodharavadhani S, Rathipriya R (2021) Non-linear neural network based forecasting model for predicting COVID-19 cases. Neural Process Lett. https://doi.org/10.1007/s11063-021-10495-w

    Article  Google Scholar 

  22. Wang Y (2017) A new concept using LSTM neural networks for dynamic system identification. In: 2017 American control conference (ACC), IEEE, pp 5324–5329

  23. Genc S (2017) Parametric system identification using deep convolutional neural networks. In: Proceedings of the 2017 international joint conference on neural networks (IJCNN17), pp. 2112–2119

  24. Lopez M, Yu W (2019) Impact of random weights on nonlinear system identification using convolutional neural networks. Inf Sci 477(1):1–14

    MathSciNet  Google Scholar 

  25. Kang Y, Chen S, Wang X, Cao Y (2018) Deep convolutional identifier for dynamic modeling and adaptive control of unmanned helicopter. IEEE Trans Neural Netw Learn Syst 30:524–538

    Article  MathSciNet  Google Scholar 

  26. Virtue P, Yu SX, Lustig M (2017) Better than real: complex-valued neural nets for MRI fingerprinting. IEEE Int Conf Image Process (ICIP) 2017:3953–3957. https://doi.org/10.1109/ICIP.2017.8297024

    Article  Google Scholar 

  27. Xiong T et al (2015) Forecasting interval time series using a fully complex-valued RBF neural network with DPSO and PSO algorithms. Inf Sci 305:77–92

    Article  Google Scholar 

  28. Dramsch J, Lüthje M, Christensen AN (2021) Complex-valued neural networks for machine learning on non-stationary physical data. Comput Geosci 146:104632 (Elsevier)

    Article  Google Scholar 

  29. Guberman N (2016) On complex valued convolutional neural networks. arXiv preprint arXiv:1602.09046

  30. Benvenuto N, Piazza F (1992) On the complex backpropagation algorithm. IEEE Trans Signal Process 40(4):967–969

    Article  Google Scholar 

  31. Hirose A, Yoshida S (2011) Comparison of complex-and real-valued feedforward neural networks in their generalization ability. In: International conference on neural information processing. Springer, Berlin, Heidelberg

  32. Hirose A (2003) Complex-valued neural networks: theories and applications, vol 5. World Scientific, Singapore

    Book  MATH  Google Scholar 

  33. Quan Y, Chen Y, Shao Y, Teng H, Xu Y, Ji H (2021) Image denoising using complex-valued deep CNN. Pattern Recognit 111:107639

    Article  Google Scholar 

  34. Zhang Z, Wang H, Xu F, Jin YQ (2017) Complex-valued convolutional neural network and its application in polarimetric SAR image classification. IEEE Trans Geosci Remote Sens 55(12):7177–7188

    Article  Google Scholar 

  35. Meyer M, Kuschk G, Tomforde, S (2020) Complex-valued convolutional neural networks for automotive scene classification based on range-beam-doppler tensors. In: 2020 IEEE 23rd International conference on intelligent transportation systems (ITSC), IEEE, pp. 1–6

  36. Yao X, Shi X, Zhou F (2020) Human activities classification based on complex-value convolutional neural network. IEEE Sens J 20(13):7169–7180

    Article  Google Scholar 

  37. Zimmermann HG et al (2011) Comparison of the complex valued and real valued neural networks trained with gradient descent and random search algorithms. In: Proc. of ESANN 2011

  38. Narendra K, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Trans Neural Netw 1(1):4–27

    Article  Google Scholar 

  39. Ruta D, Gabrys B (2007) Neural network ensembles for time series prediction. Int Joint Conf Neural Netw 2007:1204–1209

    Article  Google Scholar 

  40. Gu H, Qing G, Wang Y, Hong S, Gacanin H, Adachi F (2020) Deep complex-valued convolutional neural network for drone recognition based on RF fingerprinting

  41. Tan X, Li M, Zhang P, Wu Y, Song W (2019) Complex-valued 3-D convolutional neural network for PolSAR image classification. IEEE Geosci Remote Sens Lett 17(6):1022–1026

    Article  Google Scholar 

  42. Hongyo R, Egashira Y, Yamaguchi K (2018) Deep neural network based predistorter with ReLU activation for Doherty power amplifiers. Asia-Pacific Microw Conf (APMC) 2018:959–961

    Google Scholar 

  43. James G, Burley D (2002) Matematicas avanzadas para ingenieria. Pearson Educacion, London

    Google Scholar 

  44. Bergstra J, Bengio Y (2011) Random search for hyper-parameter optimization. J Mach Learn Res 13:281–305

    MathSciNet  MATH  Google Scholar 

  45. Box G, Jenkins G, Reinsel G (2008) Time series analysis: forecasting and control, 4th edn. Wiley, Hoboken

    Book  MATH  Google Scholar 

  46. Schoukens J, Suykens J, Ljung L (2009) Wiener-Hammerstein benchmark. In: Proceedings of the 5th IFAC symposium on system identification

  47. Ustundag BB, Kulaglic A (2020) High-performance time series prediction with predictive error compensated wavelet neural networks. IEEE Access 8:210532–210541

    Article  Google Scholar 

  48. George K, Mutalik P (2017) A multiple model approach to time-series prediction using an online sequential learning algorithm. IEEE Trans Syst, Man, Cybern: Syst 49(5):976–990

    Article  Google Scholar 

  49. Gonzalez J, Yu W (2018) Non-linear system modeling using LSTM neural networks. IFAC-PapersOnLine 51(13):485–489

    Article  Google Scholar 

  50. Shaikh MAH, Barbà K (2019) Wiener Hammerstein system identification: a fast approach through spearman correlation. IEEE Trans Instrum Measurement 68(5):1628–1636

    Article  Google Scholar 

  51. Marconato A, Schoukens J (2009) Identification of Wiener-Hammerstein benchmark data by means of support vector machines. IFAC Proc Vol 42(10):816–819

    Article  Google Scholar 

  52. Paduart J, Lauwers L, Pintelon R, Schoukens J (2012) Identification of a WienerHammerstein system using the polynomial nonlinear state space approach. Control Eng Pract 20(11):1133–1139

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departamento de Control Automático, CINVESTAV-IPN, Ciudad de México, Mexico

    Mario Lopez-Pacheco & Wen Yu

Authors
  1. Mario Lopez-Pacheco
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Wen Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Wen Yu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lopez-Pacheco, M., Yu, W. Complex Valued Deep Neural Networks for Nonlinear System Modeling. Neural Process Lett 54, 559–580 (2022). https://doi.org/10.1007/s11063-021-10644-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10644-1

Keywords