Table of Contents
About the Authors
Acknowledgments
Chapter 1: Introduction
	1.1 Numerical Computing in OCaml
	1.2 Architecture
		Basic Computing and Analytics with Owl
		Advanced Design in Owl
		Hardware and Deployment
		Research on Owl
	1.3 Summary
Chapter 2: Core Optimizations
	2.1 N-Dimensional Array in Owl
	2.2 Interface OCaml to C
	2.3 Core Optimization of Computation
		CPU Support of Parallel Computing
			Instruction-Level Parallelism
		Vectorization
			Context Switching
			Multicore Processor
		Memory
			Cache
			Prefetching
			NUMA
	2.4 Optimization Techniques
		Hardware Parallelization
		Cache Optimization
		Other Techniques
	2.5 Example: Convolution
	2.6 Automated Empirical Optimization of Software
	2.7 Summary
Chapter 3: Algorithmic Differentiation
	3.1 Introduction
		Three Ways of Differentiating
		Architecture of Algodiff Module
	3.2 Types
		Forward and Reverse Modes
			Forward Mode
				Reverse Mode
		Data Types
		Operations on AD Type
	3.3 Operators
		Calculation Rules
		Generalize Rules into Builder Template
	3.4 API
		Low-Level APIs
		High-Level APIs
	3.5 More Implementation Details
		Perturbation Confusion and Tag
		Lazy Evaluation
		Extending AD
		Graph Utility
	3.6 How AD Is Built upon Ndarray
	3.7 Summary
Chapter 4: Mathematical Optimization
	4.1 Gradient Descent
	4.2 Components
		Learning Rate
			Gradient
		Momentum
		Batch
		Checkpoint
			Params
	4.3 Gradient Descent Implementation
	4.4 Regression
		Linear Regression
		Loss
		Implementation of Linear Regression
		Other Types of Regression
		Regularization
	4.5 Summary
Chapter 5: Deep Neural Networks
	5.1 Module Architecture
	5.2 Neurons
		Core Functions
		Activation Module
	5.3 Networks
	5.4 Training
		Forward and Backward Pass
	5.5 Neural Network Compiler
	5.6 Case Study: Object Detection
		Object Detection Network Architectures
			R-CNN Architecture
			Fast R-CNN Architecture
			Faster R-CNN Architecture
			Mask R-CNN Architecture
		Implementation of Mask R-CNN
			Building Mask R-CNN
			Feature Extractor
			Proposal Generation
			Classification
	5.7 Summary
Chapter 6: Computation Graph
	6.1 The Definition of Computation Graph
		Dynamic Graph and Static Graph
		Significance in Computing
	6.2 Applications Inside the Computing System
		Basic Numerical Operations
		Algorithmic Differentiation with CGraph
		Deep Neural Network
	6.3 Design of Computation Graph Module
		Computing Device
		Types of Operation
		Shape Inference
		Creating and Linking Nodes
		Optimization of Graph Structure
		Computation Engine
	6.4 Optimizing Memory Usage in Computation Graph
	6.5 Summary
Chapter 7: Performance Accelerators
	7.1 Hardware Accelerators
		Utilizing Accelerators
	7.2 Design
		Core Abstraction
		Engines
	7.3 ONNX Engine
		Example 1: Basic Operations
		Example 2: Variable Initialization
		Example 3: Neural Network
	7.4 LaTeX Engine
	7.5 Owl Engine
	7.6 Summary
Chapter 8: Compiler Backends
	8.1 Base Library
	8.2 Backend: JavaScript
		Native OCaml
		Facebook Reason
	8.3 Backend: MirageOS
		Example: Gradient Descent
		Example: Neural Network
	8.4 Evaluation
	8.5 Summary
Chapter 9: Composition and Deployment
	9.1 Script Sharing with Zoo
		Example
		Version Control
	9.2 Service Deployment and Composition
	9.3 System Design
		Service
		Type Checking
		DSL
		Service Discovery
	9.4 Use Case
	9.5 Discussion
	9.6 Summary
Chapter 10: Distributed Computing
	10.1 Distributed Machine Learning
	10.2 The Actor Distributed Engine
		Map-Reduce Engine
		Parameter Server Engine
		Compose Actor with Owl
	10.3 Synchronization: Barrier Control Methods
	10.4 System Design Space and Parameters
		Compatibility
	10.5 Convergence Analysis
		How Effective Is Sampling
		Implementation Technique
	10.6 Evaluation
		Experiment Setup
			Accuracy
		System Progress
			Robustness to Straggler
		Sampling Settings
			Progress Inconsistency
	10.7 Summary
Chapter 11: Testing Framework
	11.1 Unit Test
	11.2 Example
	11.3 What Could Go Wrong
		Corner Cases
		Test Coverage
	11.4 Use Functor
	11.5 Performance Tests
	11.6 Summary
Appendix A: Basic Analytics Examples
	A.1 Mathematical Functions
	A.2 Linear Algebra
	A.3 Statistical Distributions
	A.4 Signal Processing
	A.5 Ordinary Differential Equation
Appendix B: System Conventions
	B.1 Pure vs. Impure
	B.2 Ndarray vs. Scalar
	B.3 Module Structures
	B.4 Infix Operators
Appendix C: Metric Systems and Constants
	C.1 Four Metric Systems
	C.2 International System of Units
Appendix D: Algodiff Module
	D.1 Operator Building
	D.2 Core Modules
	D.3 Graph Converter
Appendix E: Neural Network Module
	E.1 Graph Module
	E.2 Neuron Modules
Appendix F: Actor System for Distributed Computing
	F.1 MapReduce Engine
	F.2 Parameter Server Engine
	F.3 Peer-to-Peer Engine
Bibliography
Index