TJ
Half-day Tutorial
6/6/2017 1:00 PM
Docker Fundamentals Jumpstart
Presented by:
Ali Hussain
Flux7
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073
888-- 268-- 8770 ·· 904-- 278-- 0524 - info@techwell.com - https://www.techwell.com/
Ali Hussain
Flux7
Ali Hussain is CTO and co-founder at Flux7, an IT consulting company and Docker
partner dedicated to designing cutting-edge, scalable, and distributed systems for
innovative businesses. Ali began his career in computer architecture and
performance at ARM and Intel, and has brought this perspective to Flux7, leading
the company's research and development efforts. A highly accredited Amazon
Web Services technologist with Docker certification, Ali focuses on integrating
Docker technology into business solutions that improve operations,
development, and business productivity. Ali can be found on Twitter
@Ali_A_Hussain, via LinkedIn, or the Flux7 blog.
5/1/2017
Docker Fundamentals Jumpstart
June 6, 2017
Presenter: Ali Hussain
About Flux7
Achievements
AWS DevOps, Migration, Healthcare, and Life
Sciences Competencies
TechTarget’s “Impact Best AWS Consulting
Partner” two years in a row (2015 & 2016) “[Flux7] taught us
Flux7: Cloud and DevOps Solutions how to do 10x the
Partner Recognition Award by AWS at work in 1/10th the
Founded in 2013 reInvent 2015 time” - Patrick K, AWS
Team of 35+ Customers featured on stage at AWS re:Invent
Re:invent’14, CTO’s
Headquartered in Austin, Texas three years in a row Keynote
Docker Foundation and authorized consulting
partner
150+ happy customers through word of mouth
1
5/1/2017
Outcomes
What is Docker?
Common use Cases
Get Started: Setup a local environment with Docker
Plan
✓Docker Concepts
✓Real-world use cases
✓Docker for Development: Hands on workshop
✓Summary
2
5/1/2017
Docker Concepts
Releasing Technology
The four pillars:
Server
Infrastructure Code
Configurations
Security Rules
3
5/1/2017
Accelerating Releases
High developer Fast infrastructure
productivity provisioning
Repeatable configurations Automated policy enforcements
What tends to happen?
Startups: Enterprises:
Code first Checks and balances try to ensure
progress in lock step
Infrastructure is developed right
before launch or post-launch The entire release is slower
→ Down times, sub-optimized → The window of opportunity may be
infrastructure, low productivity missed
4
5/1/2017
Why?
Infrastructure and security policies are considered non-agile and brittle
What is needed: Treat everything as code
- Automated code pipelines (CI/CD)
- Infrastructure as code
- Configuration management
Why?
Infrastructure and security policies are considered non-agile and brittle
What is needed: Treat everything as code
- Automated code pipelines (CI/CD)
- Infrastructure as code
- Configuration management
Options: Implement the above using an array of tools, or simply consider Docker
5
5/1/2017
What is Docker?
Builds from the concept of Linux
containers, but a lot more …
Has its own file system (similar to
chroot), IP, and set of ports
Has its own process space
Docker’s attraction
Lightweight Hardware
Portable
Agnostic
6
5/1/2017
Helps with
Ineffective code pipeline management
Inconsistency across environments
Mismatches in dev and prod environments
Resource provisioning takes ages
Increasing infrastructure bills
Docker in Docker terms...
Build Once, Configure Once & Run Anywhere
7
5/1/2017
8
5/1/2017
Docker Accelerates …
Developers: Operations: Information Security:
End-to-end ownership of Application agnostic Immutable environments
application and configuration requirements, i.e., just need
to run containers robustly Smaller and more
Control networking and
consistent perimeter
provisioning
Automated analysis at
An easy path to request
build time
infrastructure
Real world use cases
9
5/1/2017
Common Use Cases
Simplifying Configuration App Isolation
Developer Productivity Code Pipeline Management
Multi-Tenancy Debugging Capabilities
Server Consolidation Rapid Deployment
Rent-A-Center
Business Challenge
Lengthy deployment cycles; non-scalable infrastructure;
numerous manual steps involved in provisioning new servers;
manual process to prepare the OS and server images.
Fortune 500 retail
organization
The company had a young DevOps department with no
technologists, and needed help and guidance in improving their
development and IT processes.
10
5/1/2017
RAC (Virtual Acceptance Now)
Outcome
A DevOps-based streamlined infrastructure that
included a high degree of automation that met security
and uptime requirements.
The solution that Flux7
RAC’s initial project has served as a proof of concept implemented for us ensures
around which they have extended their infrastructure agility in our IT operations and
development efforts, enabling us
and DevOps culture. to focus on giving our customers a
high-quality experience.
Presentation at AWS Chicago Summit 2016:
https://www.youtube.com/watch?v=JVzrGr4Fkzc
VeriFone Commerce Portal
Business Challenge
New business opportunities and a need for increased agility
helped drive a decision to migrate on-premise infrastructure to
AWS.
Fortune 1000 retail
point of sale
The company needed to ensure development was able to deliver organization
high quality, secure solutions against tight deadlines.
11
5/1/2017
VeriFone Commerce Portal
Outcome
Docker based automation tamed the project’s complex microservices, provides
highly available, portable environments to improve agility and increase fidelity
of new releases.
Invent & Simplify
Unique build - deploy system uses Docker for CI/CD, enables self-service IT for Award
dev teams.
Security is extended by making sensitive configuration data available based on re:Invent 2015
role, while ensuring development meets security processes and controls. APN Summit
Presentation at AWS Santa Clara summit 2016:
http://www.slideshare.net/AmazonWebServices/compliance-in-the-cloud-
using-security-by-design-64183932
Cars.com (auto.com)
Business Challenge
Need for agile development teams thereby reducing the
time to market and development costs
National &
International
The company also wanted to improve the end user Automotive listing
experience by significantly reducing load times website
12
5/1/2017
Cars.com (auto.com)
Outcome
Innovation was made easier since cost of being wrong is low
Organizational ownership of operations knowledge which was
very valuable for business continuity.
High fidelity and confidence in Dev and QA leading to faster The service from Flux7
and communication has
iteration of product
been the ‘Best in Class’
Video from DockerCon 2014:
https://www.youtube.com/watch?v=PBBUnNS4dRw
Web Developer Workflows
Most expensive but
least executed
Least expensive but
most executed
Developer Local Test QA Production
13
5/1/2017
Project: cars.com
Laptop
App Code
and Logs
Project: cars.com
Laptop
Local Docker Engine
14
5/1/2017
Project: cars.com
Laptop
Local Docker Engine
Docker Containers App Code
and Logs
Web App
DB Redis
Mem Log
$ Stash
Project: cars.com
Laptop
Local Docker Engine
Docker Containers App Code
and Logs
Web App
DB Redis
Mem Log
$ Stash
15
5/1/2017
Project: cars.com
Laptop
Local Docker Engine
Docker Containers App Code
and Logs
HTTP Web App
DB Redis
Mem Log
SSH
$ Stash
Project: cars.com
Laptop
Local Docker Engine
Docker Containers App Code
Shared
and Logs
folder for
HTTP Web App code and
logs
DB Redis
Mem Log
SSH
$ Stash
16
5/1/2017
Project: cars.com
Laptop
Local Docker Engine
Docker Containers App Code
Shared
and Logs
folder for run scripts on the
HTTP Web App code and laptop for
logs
interacting with
DB Redis the environment
Mem Log
SSH
$ Stash
Who sees the benefit?
Management
● Less conflict
● Faster time to market
Ops ● Happier engineers
Easier hiring
Only one standardized dev
●
environment to support
Backend Developers
Front-end HTML Devs
● Homogenous environment
● Quick visual feedback with
● Quick feedback populated data
● Agile: break it fearlessly ● Avoid installations
17
5/1/2017
Other Stories
Fugro OARS
Video from DockerCon 2016: https://www.youtube.com/watch?v=S7B-pEDoSeE
Pristine
Video from AWS Re:invent 2014:
https://www.youtube.com/watch?v=ktKzSuD9InM
Workshop
18
5/1/2017
Taxonomy: Nouns
Container Image Engine
Actual containers running the
All inclusive images that helps Lightweight container that builds
applications and includes the OS,
launch Docker containers and runs containers
user added files, and metadata
Layer Dockerfile Registry
Each file system that is File containing instructions
Application to store and distribute
stacked when Docker mounts that help automate image
Docker images
rootfs creation
Taxonomy: Verbs
Build Run Stop
The operation of building docker The action to create the first Stop a running container
images from Dockerfiles “writeable” container layer
Attach Delete
Connect to a running container Delete a container
19
5/1/2017
Taxonomy: Verbs
Commit Push Pull
Create an image from a running Push an image to a repository Pull an image from a
container repository
Taxonomy: Adjectives
Running Stopped Terminated
The container received
The container is started using The container with the
SIGTERM and SIGKILL after a
the specified command associated volume is removed
grace period
20
5/1/2017
Execution Environment
Containers
Docker engine
Operating System Kernel
The Container
From outside: From inside:
Unique IP on a Docker network Very similar to a VM. Login, run
Its own 65,536 ports shell commands, access files on
Its own root file system the file system, run ifconfig, etc.
21
5/1/2017
Docker Command Structure
docker command objects <main commands>
docker run busybox ls
docker stop mycontainer
docker start mycontainer
Docker Commands
docker pull Pull a pre-built image from the public repos
Run the container in one of 3 modes:
docker run
Background, Foreground, Interactive
docker ps List running containers
docker top Show running processes in a container
docker exec Run a command inside a container
docker attach Interact with running containers
docker cp Copy files to/from a container
22
5/1/2017
Docker Commands
docker build Build docker images from Dockerfiles
docker commit Save the container state as an image
docker images Obtain a list of all local images
docker push Push a local image to a repository
docker rmi Remove the local copy of an image
Docker Commands
docker diff List of changes in files and directories
docker logs View the logs of the running job
docker rm Remove a container
docker inspect Low-level information about containers and images
docker kill Kill the main process of the container
docker tag Tags an image with a name
23
5/1/2017
Dockerfile
Automates Image creation process
Set of instructions to create an image
General DockerFile commands’ syntax:
INSTRUCTION argument
Dockerfile Commands
MAINTAINER <author name> Set an author field for the image
ARG Build time argument
RUN <command> Execute a command in a shell or exec form
COPY <src> <destination> Copy files into containers
CMD["executable","param1
","param2"]
Provides defaults for an executing container
EXPOSE <port>; Port on which container listens to
24
5/1/2017
Dockerfile Commands
ENTRYPOINT
[‘executable’, Configure a container as an executable
‘param1’,’param2’]
WORKDIR /path/to/workdir Set Working Directory
ENV <key> <value> Set environment variables
USER <uid> Set UID for use when running an image
VOLUME [‘/data’] Enable access to a directory from a working container
LABEL “a”=”b” Apply labels to images
docker-compose.yml
A YAML file describing how a set of containers are to be run
Used to define a multi-tier app, e.g., a Wordpress container and a DB
25
5/1/2017
Hands On
1. Install Docker
1. Run a simple container
1. Run a complete website locally
Visit
https://www.katacoda.com/flux7/scenarios/l
ab
Code: https://github.com/Flux7Labs/docker-hands-on-lab
Summary
26
5/1/2017
Next Steps to Try
Develop Dockerfiles for your application
Develop docker-compose.yml files for your entire stack, i.e., app, DB, etc
Deploy the stack locally
Code Pipeline with Docker
Registry
Old service, new code:
1. Test code locally
1
Pass/Fail 2. Merge code to master branch
Dev 3. CI/CD detects new commit via
7
polling
4. Builds artifacts, container, and
4
8
2
pushes to Docker repository
3
5. Runs test on a new QA
SCM CI/CD Tool 6
environment
6. Updates cluster
7. Cluster downloads new container
5
8. Notify Developer of results
code/config
metadata
27
5/1/2017
Next Layer of Concepts
✓ Multi-host Docker orchestration (tools: Swarm, AWS ECS, Kubernetes, Mesos)
✓ Follow https://github.com/wsargent/docker-cheat-sheet as guideline for other
Docker commands to learn
✓ Third party tools: AWS Tools (BeanStalk, EC2 Container Service) and HashiCorp
tools (Nomad, Consul, and Vault)
Summary
■ Docker is emerging as a key component of future code pipelines
■ Docker accelerates the entire flow: Dev, QA, Ops, and InfoSec
■ Work usually starts at the development teams
■ Several solutions to leverage it in QA and Production
28
5/1/2017
Thank You We are hiring:
✓ DevOps Solution Architect
Aater Suleman
CEO
✓ Senior Tech Lead
Flux7.com
Austin, Texas ✓ Solutions Architect
www.flux7.com
careers@flux7.com
@flux7Labs @FutureChips
Docker in Production
29
5/1/2017
Arch Diagram
Nginx, App , Vault,
and Web Tiers
Jenkin
s
Users
NAT
Gate
way Consul
ECR CloudWatch S3 CloudFormation Cloud Trail Config
Details of Clusters
Svc 1 Svc 2 Svc 3
Svc 4 Svc 5 Svc 6
Cluster 1
30
5/1/2017
Microservices - Components
Context-based
Orchestration Service Discovery Load Balancing
routing
Logging Secret handling
Microservices - Components
Context-based
Orchestration Service Discovery Load Balancing
routing
ECS
ELB
31
5/1/2017
Code Deploy
Old service, new code:
1. Merge code to master branch
Pass/Fail Staging/Prod Cluster 2. Jenkins detects new commit via
Dev polling
4
3. Builds artifacts, container, and
pushes to Docker repository
3
7
1
5
4. Update ECS with new container
2
5. Docker updates Consul
SCM CI/CD Tool Consul 6. Consul updates Nginx
7. Notify Developer of results
6
New service:
1. Create new repo
2. Add service to Jenkins
3. Create new Docker registry
4. Push code, the above workflow
code/config runs as-is
metadata
Service Discovery
Add demo-service Typical ECS Node
Docker
Daemon
1
3
2
Container A Events
Registrator Container name/tags
Consul agent
Consul
Nginx 7
Template
6
Nginx.
https://myurl/demo-service New container added for service X
conf
32
