Table of Contents
Python for DevOps Table of Contents
Return to Python for DevOps by Noah Gift
Summary of Table of Contents
I. Python Foundations
- Chapter 2, Automating Files and the Filesystem
- Chapter 3, Working with the Command Line
II. Operations
- Chapter 4, Useful Linux Utilities
- Chapter 5, Package Management
III. Continuous Deployment
- Chapter 7, Monitoring and Logging
- Chapter 8, Pytest for DevOps
IV. Cloud Foundations
- Chapter 9, Cloud Computing
- Chapter 10, Infrastructure as Code
- Chapter 12, Container Orchestration - Kubernetes
- Chapter 13, Serverless Technologies
V. Data
- Chapter 14, MLOps and Machine learning Engineering
- Chapter 15, Data Engineering
VI. Case Studies
- Chapter 16, DevOps War Stories and Interviews
Detailed Table of Contents
Preface
- What Does DevOps Mean to the Authors?
-
- How to Contact Us
-
- Noah
- Kennedy
- Alfredo
- Grig
Python Foundations
Python Essentials for DevOps
1. Python Essentials for DevOps - Python Essentials for DevOps
-
- The Python Shell
- Procedural Programming - Procedural - Procedural Programming in Python - Python Procedural Programming
- Execution Control - Python Execution Control - Execution Control in Python - Controlling Execution - Python Controlling Execution - Controlling Execution in Python
- if/elif/else
-
- What Is an Object?
- Object Methods and Attributes
- Using Regular Expressions
- Searching
More IPython Features Using IPython to Run Unix Shell Commands
Automating Files and the Filesystem
2. Automating Files and the Filesystem - Automating Files and the Filesystem - Automating the Filesystem]] - File system
- Reading and Writing Files
Using Regular Expressions to Search Text
Dealing with Large Files
Encrypting Text Hashing with Hashlib
Encryption with Cryptography
The os Module
Managing Files and Directories Using os.path
Walking Directory Trees Using os.walk
Paths as Objects with Pathlib
3. Working with the Command Line Working with the Shell Talking to the Interpreter with the sys Module
Dealing with the Operating System Using the os Module
Spawn Processes with the subprocess Module
Creating Command-Line Tools Using sys.argv
Using argparse
Using click
Case Study: Turbocharging Python with Command-Line Tools Using the Numba Just-in-Time (JIT) Compiler
Using the GPU with CUDA Python
Running True Multicore Multithreaded Python Using Numba
4. Useful Linux Utilities Disk Utilities Measuring Performance
Retrieving Specific Device Information
Network Utilities SSH Tunneling
Benchmarking HTTP with Apache Benchmark (ab)
CPU Utilities Viewing Processes with htop
Working with Bash and ZSH Customizing the Python Shell
Searching and Replacing with Confirmation Prompts
Removing Temporary Python Files
Listing and Filtering Processes
Mixing Python with Bash and ZSH Random Password Generator
Does My Module Exist?
Changing Directories to a Module’s Path
How Fast Is this Snippet?
5. Package Management Why Is Packaging Important? When Packaging Might Not Be Needed
Packaging Guidelines Descriptive Versioning
The changelog
Choosing a Strategy
Packaging Solutions Native Python Packaging
Management with systemd Long-Running Processes
Setting It Up
Installing the Unit Log Handling
6. Continuous Integration and Continuous Deployment Real-World Case Study: Converting a Poorly Maintained WordPress Site to Hugo Setting Up Hugo
Converting WordPress to Hugo Posts
Creating an Algolia Index and Updating It
Orchestrating with a Makefile
Deploying with AWS CodePipeline
Real-World Case Study: Deploying a Python App Engine Application with Google Cloud Build
7. Monitoring and Logging Key Concepts in Building Reliable Systems
Immutable DevOps Principles Centralized Logging
Case Study: Production Database Kills Hard [Drive]]s
Did You Build It or Buy It?
Instrumentation Naming Conventions
Elasticsearch and Kibana
8. Pytest for DevOps Testing Superpowers with pytest
Getting Started with pytest Testing with pytest
Differences with unittest
The Amazing assert
Infrastructure Testing What Is System Validation?
Features and Special Fixtures
Testing Jupyter Notebooks with pytest
9. Cloud Computing Cloud Computing Foundations
Types of Cloud Computing
Types of Cloud Services Infrastructure as a Service
Virtualization and Containers Hardware Virtualization
Challenges and Opportunities in Distributed Computing
Python Concurrency, Performance, and Process Management in the Cloud Era
Process Management Manage Processes with Subprocess
Using Multiprocessing to Solve Problems
Forking Processes with Pool()
Function as a Service and Serverless
High Performance Python with Numba
Using Numba Just in Time Compiler
Using High-Performance Servers
Conclusion
10. Infrastructure as Code A Classification of Infrastructure Automation Tools
Automated Infrastructure Provisioning with Terraform Provisioning an S3 Bucket
Provisioning an SSL Certificate with AWS ACM
Provisioning an Amazon CloudFront Distribution
Provisioning a Route 53 DNS Record
Copying Static Files to S3
Deleting All AWS Resources Provisioned with Terraform
Automated Infrastructure Provisioning with Pulumi Creating a New Pulumi Python Project for AWS
Creating Configuration Values for the Staging Stack
Provisioning an ACM SSL Certificate
Provisioning a Route 53 Zone and DNS Records
Provisioning a CloudFront Distribution
Provisioning a Route 53 DNS Record for the Site URL
Creating and Deploying a New Stack
11. Container Technologies: Docker and Docker Compose What Is a Docker Container?
Creating, Building, Running, and Removing Docker Images and Containers
Publishing Docker Images to a Docker Registry
Running a Docker Container with the Same Image on a Different Host
Running Multiple Docker Containers with Docker Compose
Porting the docker-compose Services to a New Host and Operating System
12. Container Orchestration: Kubernetes Short Overview of Kubernetes Concepts
Using Kompose to Create Kubernetes Manifests from docker-compose.yaml
Deploying Kubernetes Manifests to a Local Kubernetes Cluster Based on minikube
Launching a GKE Kubernetes Cluster in GCP with Pulumi
Deploying the Flask Example Application to GKE
Installing Prometheus and Grafana Helm Charts
Destroying the GKE Cluster
13. Serverless Technologies Deploying the Same Python Function to the “Big Three” Cloud Providers Installing Serverless Framework
Deploying Python Function to AWS Lambda
Deploying Python Function to Google Cloud Functions
Deploying Python Function to Azure
Deploying a Python Function to Self-Hosted FaaS Platforms Deploying Python Function to OpenFaaS
Provisioning DynamoDB Table, Lambda Functions, and API Gateway Methods Using the AWS CDK
14. MLOps and Machine learning Engineering What Is Machine Learning? Supervised Machine Learning
Modeling
Python Machine learning Ecosystem Deep Learning with PyTorch
Cloud Machine learning Platforms
Machine learning Maturity Model Machine Learning Key Terminology
Level 1: Framing, Scope Identification, and Problem Definition
Level 2: Continuous Delivery of Data
Level 3: Continuous Delivery of Clean Data
Level 4: Continuous Delivery of Exploratory Data Analysis
Level 5: Continuous Delivery of Traditional ML and AutoML
Level 6: ML Operational Feedback Loop
Sklearn Flask with Kubernetes and Docker
Sklearn Flask with Kubernetes and Docker EDA
Modeling
Learning Assessments
15. Data Engineering Small Data Dealing with Small Data Files
Generator Pipeline to Read and Process Lines
Using YAML
Big Data Tools, Components, and Platforms Data Sources
Real-Time Streaming Ingestion]]
Case Study: Building a Homegrown Data Pipeline
Serverless Data Engineering Using AWS Lambda with CloudWatch Events
Using Amazon CloudWatch Logging with AWS Lambda
Using AWS Lambda to Populate Amazon Simple Queue Service
Wiring Up CloudWatch Event Trigger
Reading Amazon SQS Events from AWS Lambda
Conclusion
16. DevOps War Stories and Interviews Film Studio Can’t Make Film
Python Scripts Take 60 Seconds to Launch
Putting Out a Fire with a Cache and Intelligent Instrumentation
You’ll Automate Yourself Out of a Job!
DevOps Antipatterns No Automated Build Server Antipattern
Difficulties in Coordination as an Ongoing Accomplishment
No Teamwork