Table of Contents
About the Author
About the Technical Reviewer
About the Contributors
Acknowledgments
Preface
Chapter 1: Introduction to Persistent Memory Programming
	A High-Level Example Program
		What’s Different?
		The Performance Difference
		Program Complexity
		How Does libpmemkv Work?
	What’s Next?
	Summary
Chapter 2: Persistent Memory Architecture
	Persistent Memory Characteristics
	Platform Support for Persistent Memory
	Cache Hierarchy
	Power-Fail Protected Domains
	The Need for Flushing, Ordering, and Fencing
	Data Visibility
	Intel Machine Instructions for Persistent Memory
	Detecting Platform Capabilities
	Application Startup and Recovery
	What’s Next?
	Summary
Chapter 3: Operating System Support for Persistent Memory
	Operating System Support for Memory and Storage
	Persistent Memory As Block Storage
	Persistent Memory-Aware File Systems
	Memory-Mapped Files
	Persistent Memory Direct Access (DAX)
	Summary
Chapter 4: Fundamental Concepts of Persistent Memory Programming
	What’s Different?
	Atomic Updates
	Transactions
		Atomicity
		Consistency
		Isolation
		Durability
	Flushing Is Not Transactional
	Start-Time Responsibilities
	Tuning for Hardware Configurations
	Summary
Chapter 5: Introducing the Persistent Memory Development Kit
	Background
	Choosing the Right Semantics
	Volatile Libraries
		libmemkind
		libvmemcache
		libvmem
	Persistent Libraries
		libpmem
		libpmemobj
		libpmemobj-cpp
		libpmemkv
		libpmemlog
		libpmemblk
	Tools and Command Utilities
		pmempool
		pmemcheck
		pmreorder
	Summary
Chapter 6: libpmem: Low-Level Persistent Memory Support
	Using the Library
	Mapping a File
	Copying to Persistent Memory
	Separating the Flush Steps
	Summary
Chapter 7: libpmemobj: A Native Transactional Object Store
	What is libpmemobj?
	Why not malloc()?
	Grouping Operations
	Memory Pools
		Creating Memory Pools
		Pool Object Pointer (POP) and the Root Object
		Opening and Reading from Memory Pools
	Memory Poolsets
		Concatenated Poolsets
		Replica Poolsets
	Managing Memory Pools and Poolsets
	Typed Object Identifiers (TOIDs)
	Allocating Memory
	Persisting Data
		Atomic Operations
		Reserve/Publish API
		Transactional API
		Optional Flags
		Persisting Data Summary
	Guarantees of libpmemobj's APIs
	Managing Library Behavior
	Debugging and Error Handling
	Summary
Chapter 8: libpmemobj-cpp: The Adaptable Language - C++ and Persistent Memory
	Introduction
	Metaprogramming to the Rescue
		Persistent Pointers
		Transactions
		Snapshotting
		Allocating
	C++ Standard limitations
		An Object’s Lifetime
		Trivial Types
		Object Layout
		Pointers
		Limitations Summary
	Persistence Simplified
	The Ecosystem
		Persistent Containers
		Examples of Persistent Containers
	Summary
Chapter 9: pmemkv: A Persistent In-Memory Key-Value Store
	pmemkv Architecture
	A Phonebook Example
	Bringing Persistent Memory Closer to the Cloud
	Summary
Chapter 10: Volatile Use of Persistent Memory
	Introduction
	Background
		Memory Allocation
		How it Works
		Supported “Kinds” of Memory
	The memkind API
		Kind Management API
			Kind Creation
				Creating a Fixed-Size Heap
				Creating a Variable Size Heap
			Detecting the Memory Kind
				Automatic Kind Detection
				Memory Kind Detection
			Destroying Kind Objects
		Heap Management API
			Allocating Memory
			Freeing Allocated Memory
		Kind Configuration Management
			Memory Usage Policy
		Additional memkind Code Examples
	C++ Allocator for PMEM Kind
		pmem::allocator methods
		Nested Containers
	C++ Examples
		Using the pmem::allocator
		Creating a Vector of Strings
	Expanding Volatile Memory Using Persistent Memory
	libvmemcache: An Efficient Volatile Key-Value Cache for Large-Capacity Persistent Memory
		libvmemcache Overview
		libvmemcache Design
			Extent-Based Allocator
			Scalable Replacement Policy
		Using libvmemcache
	Summary
Chapter 11: Designing Data Structures for Persistent Memory
	Contiguous Data Structures and Fragmentation
		Internal and External Fragmentation
		Atomicity and Consistency
			Transactions
			Copy-on-Write and Versioning
		Selective Persistence
		Example Data Structures
			Hash Table with Transactions
			Hash Table with Transactions and Selective Persistence
			Sorted Array with Versioning
	Summary
Chapter 12: Debugging Persistent Memory Applications
	pmemcheck for Valgrind
		Stack Overflow Example
		Memory Leak Example
	Intel Inspector – Persistence Inspector
		Stack Overflow Example
		Memory Leak Example
	Common Persistent Memory Programming Problems
		Nonpersistent Stores
		Stores Not Added into a Transaction
		Memory Added to Two Different Transactions
		Memory Overwrites
		Unnecessary Flushes
		Out-of-Order Writes
	Summary
Chapter 13: Enabling Persistence Using a Real-World Application
	The Database Example
	Different Persistent Memory Enablement Approaches
	Developing a Persistent Memory-Aware MariaDB* Storage Engine
		Understanding the Storage Layer
		Creating a Storage Engine Class
			Creating a Database Table
			Opening a Database Table
			Closing a Database Table
			INSERT Operation
			UPDATE Operation
			DELETE Operation
			SELECT Operation
	Summary
Chapter 14: Concurrency and Persistent Memory
	Transactions and Multithreading
	Mutexes on Persistent Memory
	Atomic Operations and Persistent Memory
		Lock-Free Algorithms and Persistent Memory
	Concurrent Data Structures for Persistent Memory
		Concurrent Ordered Map
			Find Operation
			Insert Operation
			Erase Operation
		Concurrent Hash Map
			Find Operation
			Insert Operation
			Erase Operation
	Summary
Chapter 15: Profiling and Performance
	Introduction
	Performance Analysis Concepts
		Compute-Bound vs. Memory-Bound
		Memory Latency vs. Memory Capacity
		Read vs. Write Performance
		Memory Access Patterns
		I/O Storage Bound Workloads
	Determining the Suitability of Workloads for Persistent Memory
		Volatile Use Cases
			Identifying Workloads That Are Memory-Capacity Bound
			Identifying the Hot Working Set Size of a Workload
		Use Cases Requiring Persistence
	Performance Analysis of Workloads Using Persistent Memory
		Characterizing the Workload
		Memory Bandwidth and Latency
		Persistent Memory Read-Write Ratio
		Working Set Size and Memory Footprint
		Non-Uniform Memory Architecture (NUMA) Behavior
			Tuning the Hardware
			Addressable Memory Capacity
			Bandwidth Requirements
			BIOS Options
		Optimizing the Software for Persistent Memory
			Guided Data Placement
			Memory Access Optimization
			NUMA Optimizations
				Data Allocation vs. First Access
				Thread Migration
			Large and Huge Pages
	Summary
Chapter 16: PMDK Internals: Important Algorithms and Data Structures
	A Pool of Persistent Memory: High-Level Architecture Overview
	The Uncertainty of Memory Mapping: Persistent Memory Object Identifier
	Persistent Thread Local Storage: Using Lanes
	Ensuring Power-Fail Atomicity: Redo and Undo Logging
		Transaction Redo Logging
		Transaction Undo Logging
		libpmemobj Unified Logging
	Persistent Allocations: The Interface of a Transactional Persistent Allocator
	Persistent Memory Heap Management: Allocator Design for Persistent Memory
	ACID Transactions: Efficient Low-Level Persistent Transactions
	Lazy Reinitialization of Variables: Storing the Volatile State on Persistent Memory
	Summary
Chapter 17: Reliability, Availability, and Serviceability (RAS)
	Dealing with Uncorrectable Errors
		Consumed Uncorrectable Error Handling
		Unconsumed Uncorrectable Error Handling
			Patrol Scrub
			Unconsumed Uncorrectable Memory-Error Persistent Memory Root-Device Notification
			Address Range Scrub
		Clearing Uncorrectable Errors
	Device Health
		ACPI-Defined Health Functions (_NCH, _NBS)
		Vendor-Specific Device Health (_DSMs)
		ACPI NFIT Health Event Notification
	Unsafe/Dirty Shutdown
		Application Utilization of Data Loss Count (DLC)
	Summary
Chapter 18: Remote Persistent Memory
	RDMA Networking Protocols
	Goals of the Initial Remote Persistent Memory Architecture
	Guaranteeing Remote Persistence
		General-Purpose Remote Replication Method
			How Does the General-Purpose Remote Replication Method Make Data Persistent?
			Performance Implications of the General-Purpose Remote Replication Method
		Appliance Remote Replication Method
			How Does the Appliance Remote Replication Method Make Data Persistent?
			Performance Implications of the Appliance Remote Replication Method
	General Software Architecture
	librpmem Architecture and Its Use in Replication
		Configuring Remote Replication Using Poolsets
		Performance Considerations
		Remote Replication Error Handling
		Say Hello to the Replicated World
			Execution Example
	Summary
Chapter 19: Advanced Topics
	Nonuniform Memory Access (NUMA)
		NUMACTL Linux Utility
		NDCTL Linux Utility
		Intel Memory Latency Checker Utility
		NUMASTAT Utility
		Intel VTune Profiler – Platform Profiler
		IPMCTL Utility
		BIOS Tuning Options
		Automatic NUMA Balancing
	Using Volume Managers with Persistent Memory
	The mmap() MAP_SYNC Flag
	Summary
Appendix A: How to Install NDCTL and DAXCTL on Linux
	Prerequisites
	Installing NDCTL and DAXCTL Using the Linux Distribution Package Repository
		Searching for Packages Within a Package Repository
		Installing NDCTL and DAXCTL from the Package Repository
			Installing PMDK on Fedora 22 or Later
			Installing PMDK on RHEL and CentOS 7.5 or Later
			Installing PMDK on SLES 12 and OpenSUSE or Later
			Installing PMDK on Ubuntu 18.04 or Later
Appendix B: How to Install the Persistent Memory Development Kit (PMDK)
	PMDK Prerequisites
	Installing PMDK Using the Linux Distribution Package Repository
		Package Naming Convention
		Searching for Packages Within a Package Repository
		Installing PMDK Libraries from the Package Repository
			Installing PMDK on Fedora 22 or Later
			Installing PMDK on RHEL and CentOS 7.5 or Later
			Installing PMDK on SLES 12 and OpenSUSE or Later
			Installing PMDK on Ubuntu 18.04 or Later
	Installing PMDK on Microsoft Windows
Appendix C: How to Install IPMCTL on Linux and Windows
	IPMCTL Linux Prerequisites
		libsafec
	IPMCTL Linux Packages
	IPMCTL for Microsoft Windows
	Using ipmctl
Appendix D: Java for Persistent Memory
	Volatile Use of Persistent Memory
		Heap Allocation on Alternative Memory Devices
			Partial Heap Allocation on Alternative Memory Devices
			Non-volatile Mapped Byte Buffers
	Persistent Collections for Java (PCJ)
		Using PCJ in Java Applications
	Low-Level Persistent Library (LLPL)
		Using LLPL in Java Applications
	Summary
Appendix E: The Future of Remote Persistent Memory Replication
Glossary
Index