Skip to main content

Deep Q-Learning

  • Chapter
  • First Online:
Deep Reinforcement Learning with Python
  • 5050 Accesses

Abstract

In this chapter, we will do a deep dive into Q-learning combined with function approximation using neural networks. Q-learning in the context of deep learning using neural networks is also known as Deep Q Networks (DQN). We will first summarize what we have talked about so far with respect to Q-learning. We will then look at code implementations of DQN on simple problems followed by training an agent to play Atari games. Following this, we will extend our knowledge by looking at various modifications that can be done to DQN to improve the learning, including some very recent and state-of-the-art approaches. Some of these approaches may involve a bit of math to understand the rationale for these approaches. However, we will try to keep the math to a minimum and include just the required details to appreciate the background and reasoning. All the examples in this chapter will be coded using either the PyTorch or TensorFlow library. Some code walk-throughs will have the code for both PyTorch and TensorFlow, while others will be discussed using only PyTorch.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

eBook
USD 18.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://arxiv.org/pdf/1312.5602.pdf

  2. 2.

    https://arxiv.org/pdf/1511.05952.pdf

  3. 3.

    https://arxiv.org/pdf/1509.06461v3.pdf

  4. 4.

    https://arxiv.org/pdf/1511.06581.pdf

  5. 5.

    https://arxiv.org/pdf/1706.10295.pdf

  6. 6.

    https://arxiv.org/pdf/1707.06887.pdf

  7. 7.

    https://arxiv.org/pdf/1710.10044.pdf

  8. 8.

    https://arxiv.org/pdf/1707.01495.pdf

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Nimish Sanghi

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Sanghi, N. (2021). Deep Q-Learning. In: Deep Reinforcement Learning with Python. Apress, Berkeley, CA. https://doi.org/10.1007/978-1-4842-6809-4_6

Download citation

Publish with us

Policies and ethics