Reinforcement Learning for Finance: Solve Problems in Finance with CNN and RNN Using the TensorFlow Library

Table of Contents
About the Author
Acknowledgments
Preface
Introduction
Chapter 1: Overview
	1.1 Methods for Training Neural Networks
	1.2 Machine Learning in Finance
	1.3 Structure of the Book
Chapter 2: Introduction to TensorFlow
	2.1 Tensors and Variables
	2.2 Graphs, Operations, and Functions
	2.3 Modules
	2.4 Layers
	2.5 Models
	2.6 Activation Functions
	2.7 Loss Functions
	2.8 Metrics
	2.9 Optimizers
	2.10 Regularizers
	2.11 TensorBoard
	2.12 Dataset Manipulation
	2.13 Gradient Tape
Chapter 3: Convolutional Neural Networks
	3.1 A Simple CNN
	3.2 Neural Network Layers Used in CNNs
	3.3 Output Shapes and Trainable Parameters of CNNs
	3.4 Classifying Fashion MNIST Images
	3.5 Identifying Technical Patterns in Security Prices
	3.6 Using CNNs for Recognizing Handwritten Digits
Chapter 4: Recurrent Neural Networks
	4.1 Simple RNN Layer
	4.2 LSTM Layer
	4.3 GRU Layer
	4.4 Customized RNN Layers
	4.5 Stock Price Prediction
	4.6 Correlation in Asset Returns
Chapter 5: Reinforcement Learning Theory
	5.1 Basics
	5.2 Methods for Estimating the Markov Decision Problem
	5.3 Value Estimation Methods
		5.3.1 Dynamic Programming
			Finding the Optimal Path in a Maze
			European Call Option Valuation
			Valuation of a European Barrier Option
		5.3.2 Generalized Policy Iteration
			Policy Improvement Theorem
			Policy Evaluation
			Policy Improvement
		5.3.3 Monte Carlo Method
			Pricing an American Put Option
		5.3.4 Temporal Difference (TD) Learning
			SARSA
			Valuation of an American Barrier Option
			Least Squares Temporal Difference (LSTD)
			Least Squares Policy Evaluation (LSPE)
			Least Squares Policy Iteration (LSPI)
			Q-Learning
			Double Q-Learning
			Eligibility Trace
		5.3.5 Cartpole Balancing
	5.4 Policy Learning
		5.4.1 Policy Gradient Theorem
		5.4.2 REINFORCE Algorithm
		5.4.3 Policy Gradient with State-Action Value Function Approximation
		5.4.4 Policy Learning Using Cross Entropy
	5.5 Actor-Critic Algorithms
		5.5.1 Stochastic Gradient–Based Actor-Critic Algorithms
		5.5.2 Building a Trading Strategy
		5.5.3 Natural Actor-Critic Algorithms
		5.5.4 Cross Entropy–Based Actor-Critic Algorithms
Chapter 6: Recent RL Algorithms
	6.1 Double Deep Q-Network: DDQN
	6.2 Balancing a Cartpole Using DDQN
	6.3 Dueling Double Deep Q-Network
	6.4 Noisy Networks
	6.5 Deterministic Policy Gradient
		6.5.1 Off-Policy Actor-Critic Algorithm
		6.5.2 Deterministic Policy Gradient Theorem
	6.6 Trust Region Policy Optimization: TRPO
	6.7 Natural Actor-Critic Algorithm: NAC
	6.8 Proximal Policy Optimization: PPO
	6.9 Deep Deterministic Policy Gradient: DDPG
	6.10 D4PG
	6.11 TD3PG
	6.12 Soft Actor-Critic: SAC
	6.13 Variational Autoencoder
	6.14 VAE for Dimensionality Reduction
	6.15 Generative Adversarial Networks
Bibliography
Index