Jenkins Introduction: Jenkins Is The "Leading Open Source Automation Server."
Jenkins Introduction: Jenkins Is The "Leading Open Source Automation Server."
https://jenkins.io/
• As is explained above on their homepage, Jenkins is an open source automation
server.
• It's built in Java.
• It's basically a platform for the Software Development Life Cycle (SDLC).
• It's commonly used to implement continuous integration and continuous delivery
concepts
specifically.
• It's useful to anyone who works on a software project, including:
1. Application Developers
2. Web Developers
3. IT Operations
4. Systems Engineers
5. DevOps Engineers
Jenkins is known for being easy to use and highly adaptable. There are thousands
of customizations and
plugins that make Jenkins suitable for most use cases. It also works in the most
common OS environments
and is relatively lightweight (doesn't use a lot of resources).
General Terms
Below are some commonly used abbreviations and terms that you'll notice used in
the course videos and
the rest of this guide:
• CD: CD stands for Continuous Delivery unless specified otherwise. It could also be
Continuous
Deployment.
• CI: Continuous Integration
• DevOps: It's really an organizational culture. The concept of developers and
operations working in
harmony. CI/CD is a common strategy used, but DevOps encompasses much more.
• DSL: Domain Specific Language
• GUI: Graphical User Interface
• Job: A task configured in Jenkins; It's synonymous with Project. "Job" is now the
deprecated term.
• JSON: lightweight data-interchange format that uses Key/Value pairs
• md5sum: The digital fingerprint of a file (128-bit MD5 hashes)
• Pipeline: The job type created buy the Pipeline plugin; It's used more generically
in CI/CD
discussion.
Project: A task configured in Jenkins; It's synonymous with Job. "Job" is now the
deprecated term.
• Repo: Repository
4-
• Have a common mainline Branch (usually master).
• Automate the Build.
• Minimize potential for user error, automate everything possible
• Make the Build Self-testing.
• Self-testing code
• Use Unit Tests for granular functionality.
• Everyone commits frequently (at least daily, preferably).
• Communication is key! Seems basic, but it's where many orgs fail.
• Frequent merges will help avoid conflict errors.
• The working branch should be updated as frequently as possible to help avoid
very large diffs
while merging.
• Build every commit.
• Prioritize fixing broken builds.
• Keep your builds fast!
Installing/Configuring Jenkins
Installation Wizard
• Utility in WebGUI that allows for some basic configuration and the installation of
common plugins
• After installing the package and starting the service via the CLI, the installation
wizard becomes
available from the Jenkins WebGUI.
• http://your-server-fqdn:8080/
• Secret password in /var/lib/jenkins/secrets/initialAdminPassword
• Can add an admin user and set credentials
Prerequisite Install
1. Go to: http://www.oracle.com/technetwork/java/javase/downloads/jdk8-
downloads-2133151.html
2. Download the appropriate Java jdk version from the Oracle website. For this
course we use jdk-8u121.
3. Copy the package from your local environment to the target server.
6-
4. Install the jdk package.
rpm -Uvh jdk-8u121-linux-x64.rpm
5. Set up Alternatives for Java:
alternatives --install /usr/bin/java java /usr/java/latest/bin/java
200000
alternatives --install /usr/bin/javac javac /usr/java/latest/bin/javac
200000
alternatives --install /usr/bin/jar jar /usr/java/latest/bin/jar 200000
• Note: Check out Terry's nugget on this for more detail: "Setting local and global
java
environment variables"
6. Set JAVA_HOME environmental variable in rc.local.
vi /etc/rc.local
export JAVA_HOME="/usr/java/latest"
Jenkins Install
1. Add the Jenkins repo to your yum sources on the CentOS node.
wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhatstable/
jenkins.repo
2. Import the Jenkins rpm signing key.
rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
3. Install the Jenkins package.
• We will be using version 2.19.4 which is the target version for the certification
test.
yum install -y jenkins-2.19.4-1.1
4. Check for services running on 8080 before starting the Jenkins service.
netstat -tulpn | grep 8080
5. If nothing is running on 8080, go ahead and start the service via systemctl.
7-
systemctl start jenkins
6. Also, enable the Jenkins service so it starts on system startup.
systemctl enable jenkins
7. Check again for services running on port 8080.
• There will be a slight delay, so we'll use watch to wait for the signal.
watch n=1 "netstat -tulpn | grep 8080"
• ctrl-c to break the watch when the service is shown to be running on 8080
8. Visit the WebGUI at: http://your-server-fqdn:8080/
9. You'll be prompted for the initialAdminPassword which is located in
/var/lib/jenkins/secrets/
initialAdminPassword on the system being configured. You'll want to cat that and
copy and paste it
into the browser.
cat /var/lib/jenkins/secrets/initialAdminPassword
10. Paste the password into Install Wizard.
8-
15. Now you have your Jenkins Master up and running!
Configuring Jenkins
Plugins
Plugins add additional functionality to Jenkins.
• Can be used to add functionality in many ways
• "Credentials" manages user Credentials.
• "Git" and "GitHub" manage Git integrations.
• "JUnit" handles unit testing reports.
• There are thousands of plugins.
• Plugins (.hpi files) of various releases can be downloaded from:
• http://updates.jenkins-ci.org/download/plugins
Plugin Manager
The UI to install and configure plugins in the Jenkins WebGUI
Procedure for Installing Plugins
1. From the Dashboard, click "Manage Jenkins."
2. Click "Manage Plugins."
3. Click the "Available" tab.
4. Filter plugins for the targeted plugin with the search box.
5. Click the "Install" button for the desired plugin.
Procedure for Removing Plugins
1. From the Dashboard, click "Manage Jenkins."
2. Click "Manage Plugins."
3. Click the "Installed" tab.
4. Click the "Uninstall" button for the desired plugin.
Procedure for Downgrading Plugins
9-
1. From the Dashboard, click "Manage Jenkins."
2. Click "Manage Plugins."
3. Click the "Installed" tab.
4. Click the "Downgrade to..." button for the desired plugin.
Procedure for Upgrading Plugins
1. From the Dashboard, click "Manage Jenkins."
10 -
• Build History
• You'll see an overview display of build history for all projects in graphical form.
• Manage Jenkins
• Where Jenkins is managed
• My Views
• For configuring custom views for projects for the logged in user.
• Credentials
• Lists credentials that have been configured for Jenkins
Build Queue
Jobs waiting for an executor are listed here.
Build Executor Status
• The status of projects associated with Jenkins executors
• An executor runs projects dictated by Jenkins.
• They can run in parallel.
• The default number of executors on the master is 2.
Build History
• Shows the history of builds
• Stable is blue
• Red is broken
Security
11 -
Matrix Security
• Allows you to grant specific permissions to users and groups.
• Most Common Permissions: Overall, Slave, Job, Run, View, and SCM.
Configuration Procedure
1. Go to the Jenkins Dashboard.
http://<your-server>:8080
2. Click "Manage Jenkins."
3. Click "Configure Global Security."
4. Check "Enable Security."
5. Select "Jenkins' own user database."
6. Place a check mark next to "Allow users to sign up."
7. Select "Matrix-based Security."
8. Select the admin user and check all the permissions.
9. Click Save.
10. (Optional) Set "Read" permissions for "Anonymous."
11. (Optional) Set options appropriate for other users.
Roles/Permissions
Categories and Associated Permissions
Overall
• Administer:
• Make system-wide configuration changes.
• Perform highly sensitive operations that amounts to full local system access
(within the scope
granted by the underlying OS)
• Read:
• View almost all pages within Jenkins
12 -
• RunScripts:
• Run groovy scripts via the groovy console or groovy cli command
• UploadPlugins:
• Upload arbitrary plugins
• ConfigureUpdateCenter:
• Configure update sites and proxy settings
Slave
• Configure:
• Configure existing slaves
• Delete:
• Delete existing slaves
• Create:
13 -
have permission to view jobs
• Build:
• Start a new build and cancel a running build
• Workspace:
• Retrieve the contents of a workspace that Jenkins has checked out for performing
a build
• Cancel:
• Cancel a running build
Run
• Delete:
• Delete specific builds from a build's history
• Update:
• Update the description and other properties of a build
• For example, to leave notes about the cause of a build failure
View
• Create:
• Create new views
• Delete:
• Delete existing views
• Configure:
• Update the configuration of existing views
• Read:
• See any existing views
SCM
• Tag: Create a new tag in the source code repository for a given build
Jobs/Projects
14 -
Organizing Projects
Folders
• Creates a container with nested items
• It creates a separate namespace for the nested projects.
• Projects can be organized in folders.
Navigation
Left Panel -> New Item -> Folder
Adding Items to Folders
• 3 Options:
• Hover and click arrow on Existing Item; choose "move"; select target folder.
• Click existing item; In the item view page, select "Move" in the left panel; select
target folder.
• For new items Click folder; in the folder view, select new item; it will
automatically be added
to that folder.
Configuration
• Health Metrics "Child item with worst health"
• The item in the folder with the worst "health" dictates the health of the folder.
• If you set recursive, item's in nested sub-folders are used to calculate this health.
Views
A view is simply a view of items configured in Jenkins.
• The Dashboard has a default view, "All"
• Can be configured for a user (using My Views) or system-wide
• Add a new view by clicking the "+" next to the tabs in the table on the homepage
or on the "My
Views" page.
Navigation
• System-wide views can be accessed with tabs on the Dashboard home page.
• User views, can be accessed with the "My Views" button in the left panel of the
home page.
My View
15 -
• Shows all projects that the user has access to
List View
• Shows projects in a simple list format.
• You can arbitrarily filter which jobs show in this view.
Configuration
• Filter Build Queue/Executors
• You can filter the projects that show in the Build Queue/Executor panels based on
the filters set
in this view.
• Status filter
• Filter based on the current status of the job
Parameterized Projects
• These projects accept user parameters. Parameters can be added during job
configuration.
• When triggering a build, one can select "Build with Parameters" and specify
parameter values.
• Parameters are exposed as environment variables to scripts, and can also be
accessed from Project
configuration.
• Invocation Example:
"${branch}"
Configuration
• From Project Configuration, select "This project is parameterized."
16 -
Parameter Types
• String Parameter: The most common, generic parameter; simply a string
• File Parameter: A file as a parameter; it should be in the workspace, as part of a
source code
management checkout
• Boolean Parameter: A true/false value
• Choice Parameter: dropdown list, first line is the default value
• Credentials Parameter: Credential as a parameter; UUID is exposed
• List Subversion tags: Tags in a subversion repo
• Multi-line String Parameter: Like a string parameter, but multi-line
• Password Parameter: A password that can be set as a string value; password is
protected when
input in "Build with parameters" view. Otherwise, it's accessed like a string
parameter. The value is not
protected in the console view.
• Run Parameter:
• Access information from a specified build of a project
• To use a Run Parameter, the value should be in the format:
jobname#buildNumber (eg. "&MyRunParam=foo-job%2399" for foo-job #99)
Types of Projects
17 -
Allows for management of job execution in another automation system
Multi-Configuration Project
• For projects that have a lot of different configurations
• It can be best if you need to test in multiple different types of environments.
• This is likely more suitable for projects that are comprised of many build steps.
Folder
• Creates a container with nested items
• It creates a separate namespace for the nested projects.
• Projects can be organized in folders.
GitHub Organization
Scans a GitHub organization for repositories and branches
Multibranch Pipeline
Detects branches in an SCM repository
Upstream Versus Downstream Projects
• A project that is triggered by another project is considered to be downstream of
that project.
• A project that triggers another project is considered to be upstream of that
project.
Builds
A build can describe what is produced from a project, or a single run of a project.
Build Terms
• Artifact: Something produced from a build
• Build Step: A single action in a build
• Repository: The home of the sources code
• Slave: An agent or node that runs builds
• Trigger: A mechanism that starts a build
Build Steps
• Build steps are added as part of the project configuration.
18 -
• They can be specified and configured from a Jenkinsfile in pipeline configurations.
Examples of Build Steps:
• Execute shell can be a groovy script
• Execute a command in a Docker container
• Execute an Ant/Maven Build
19 -
• Described as a "pull" mechanism as opposed to "push."
• If there is a change for the specified ref(s) or branch(es), the project will run.
• If using the Jenkins Git plugin with GitHub, you can select this field and leave it
empty; the
Githook will trigger builds on changes. This is actually using the SCM Polling setting
for a "push"
scenario.
• The interval setting uses the same cron format as described in the "Time Interval"
trigger below.
Build Periodically
• Builds on a pre-defined interval
• Schedule Config Item
• Same cron format is used for "SCM Polling"
• "H" stands for hash.
• The H symbol can be used with a range. For example,
H H(0-7) * * *
20 -
45 9-16/2 * * 1-5
# once in every two hours slot between 9 AM and 5 PM every weekday
(perhaps at 10:38 AM, 12:38 PM, 2:38 PM, 4:38 PM)
H H(9-16)/2 * * 1-5
# once a day on the 1st and 15th of every month except December
H H 1,15 1-11 *
# Runs every year
@yearly
# OR
@annually
# Runs every month
@monthly
# Runs every week
@weekly
# Runs every day
@daily
# Runs everyday some time between 12:00am to 12:59am
@midnight
# Runs every hour
@hourly
21 -
• Git hooks can be configured in the more traditional way as well, by adding scripts
to the ".git/
hooks" directory in the repo path.
• Both the GitHub and Git plugins provide a hook inherently.
• Prerequisites for the procedures below:
• Have SSH keys configured for Jenkins and GitHub
• Your Jenkins master must be publicly routable, reachable from the central GitHub
servers.
• The Project must be previously configured to monitor the GitHub project already
in the "Source
Code Management" section.
Procedure for Configuring the GitHub Plugin Git Hook
1. Log in to GitHub.
2. Within GitHub, Click your project.
3. Click "Settings" from the project view.
4. Click "Integrations and Services."
5. Find "Jenkins (GitHub plugin)", via search.
6. Set the following as the "Jenkins hook url":
http://<your-server-hostname>.<your-domain>:<port>/github-webhook/
# e.g. http://brandon4231.mylabserver.com:8080/github-webhook/
7. On your Jenkins Master Dashboard, log in to Jenkins with Administrator User.
8. Click your project.
9. Click 'Configure' from the left panel.
10. Under build triggers, check "GitHub hook trigger for GITScm polling."
11. Click "Save."
12. (Optional) Make a commit to the repo and wait for the build to trigger to test.
13. (Optional) On GitHub: From the "Jenkins (GitHub plugin)" view in the
"Integrations and Services"
section, there is an option to test. Click "Test Service" from there and it will attempt
to trigger a build from
there without committing to the repo.
Procedure for Configuring the Git Plugin Git Hook
22 -
1. Log in to GitHub.
2. On GitHub, click your project.
3. Click "Settings" from the project view.
23 -
Copying Artifacts
Can leverage the "Copy Artifact" Plugin to move artifacts between projects/jobs
Procedure for Copying Artifacts
1. Install the "Copy Artifact" plugin.
2. In the "Project Configuration" view, go to the "Build" section.
3. Click the "Add build step" dropdown, and select "Copy artifacts from another
project."
4. Insert the "Project name" of the project you'd like to copy the artifact from.
5. Select the appropriate option for "Which build."
• "Latest successful build" is the default.
6. Select the "Artifacts to copy" or "Artifacts not to copy."
7. Specify the "Target directory" from within the workspace for the artifacts to land.
Setting Artifact Retention Policy
1. In the "Project Configuration" view, go to the "General" section.
2. Check "Discard old builds."
3. Click "Advanced."
4. Specify "Days to keep artifacts."
5. Specify "Max # of builds to keep with artifacts."
Fingerprints
Tracks md5sum of artifacts that are used between projects.
Viewing Fingerprints of Artifacts
1. In the "Project Build" view, click "See Fingerprints."
2. From there you can see the Artifacts and their original owning projects.
3. If you click "More Details," you can see everywhere where that specific file has
24 -
Distributed Builds
• Running builds on one or more slaves instead of a master-only configuration.
• The master is the controller, while the slaves run projects.
• Distributing builds is a way to lower the resource load on the master.
• As the scope of a Jenkins configuration expands with more and more projects, it
may make sense to
incrementally add build slaves.
About Build Slaves
• Synonymous with Build Agent
• Triggered by ssh agent by default
• SSH, JNLP and Cloud are the different agent types.
• Once set up, the distribution of tasks between nodes is fairly automatic.
• Adding slaves doesn't really change how you interact with Jenkins much at all.
• The master will still serve all administrative tasks, http requests.
Executors
Executors dictate how many builds can run on a node at a given time.
• Executors can still exist on the master after you configure build slaves.
• "# of executors" can be set by selecting the configuration icon (gear) in the
"Manage Nodes" list
view for the node, and then setting the number in the corresponding field.
Assigning Nodes to Projects
Specific build slaves can be selected for projects with expression matching in the
"Project Configuration"
view, or in the Jenkinsfile for Pipeline applications.
Labels
• Expressions can match on the node's name, which is defined in "Manage Nodes"
or by label.
• Labels can be added by selecting the configuration icon (gear) in the "Manage
Nodes" list view for
the node, and then adding a space separated list of labels to the "Labels" field.
• Examples:
CentOS JDK linux
Monitoring Build Slaves
25 -
• Agents can be monitored from the "Manage Nodes" view.
• A disconnected node will show a small red 'x' on it.
• You can also view the following status items:
• Architecture
• Clock Difference ("In sync" is desirable)
• Free disk space
• Free swap space
• Free temp space
• Response time (Generally, the lower the better)
26 -
5. Paste the key in the file vim. Save with ":wq".
• From the Jenkins Dashboard
6. Click "Manage Jenkins" from the left panel.
7. Click "Manage Nodes."
8. Click "Add Node."
9. Set a name for your node (e.g. "Slave 1").
10. Select "Permanent Node."
11. Set "Remote root directory" to /var/lib/jenkins.
12. Set "Usage" to "Use this node as much as possible."
13. Set "Launch Method" to "Launch slave agents via SSH."
14. Set "Host" to your node's FQDN (e.g. brandon4232.mylabserver.com).
15. Select "Add" under "Credentials."
16. Set "Kind" to "SSH Username with private key."
17. Set "Username" to "jenkins."
18. Set "Private key" to "From the Jenkins Master."
19. Click "Add."
20. Choose the new credential from the "Credentials" dropdown.
21. Click "Save."
22. The agent should now be available for use.
Build Tools Configuration
Often times, plugins or tools have specific configuration.
Navigation
Dashboard -> Manage Jenkins -> Global Tool Configuration
OR
Dashboard -> Manage Jenkins -> Configure System
Configurable Build Tool Examples
• Ant
27 -
28 -
2. Install the git client RPM.
yum install -y git
The Git Plugin
The Git plugin allows the integration of Git functionality within a project.
Environment Variables
The Git plugin sets several environment variables that are accessible from build
steps and configuration:
• GIT_AUTHOR_EMAIL and GIT_COMMITTER_EMAIL: The email entered if the
"Custom
user name/e-mail address" behaviour is enabled; falls back to the value entered in
the Jenkins system
config under "Global Config user.email Value" (if any)
• GIT_AUTHOR_NAME and GIT_COMMITTER_NAME: The name entered if the
"Custom
user name/e-mail address" behaviour is enabled; falls back to the value entered in
the Jenkins system
config under "Global Config user.name Value" (if any)
29 -
Incremental Updates Versus Clean Check Out
• An incremental change is usually much faster as the initial environment is already
set up and
available.
• However, you can't be fully assured that the environment is clean, or has
remnants of the previous
build job.
• A clean check out ensures that the environment is untouched.
Checking In Code
• Users should check in daily to meet CI best practices.
• The developer should also check for functionality in the local development
environment against the
most recent mainline code set before checking in changes.
Infrastructure As Code
• Infrastructure is completely defined in code.
• The state of all systems should be defined in code as opposed to local
configuration files on the
systems themselves.
• Modern config management tools like Puppet, Chef, and Ansible make this
possible, along with
automated CI/CD tools like Jenkins.
Testing
• Automated testing with Jenkins is fast.
• Shortening the feedback loop makes developers more productive.
Unit Tests
• Tests a small part of the code set
• Usually associated with an individual class method if applicable
30 -
• Unit test failures can be set to trigger a Build Breakage or not.
JUnit Example
Ensures that the "Rectangle" instance function "getArea" returns '6'
// A JUnit Test
Rectangle myRectangle = new Rectangle(2,3);
.. .
@Test
public void testGetArea() {
assertEquals(myRectangle.getArea(), 6);
}
Reporting
• You can publish reports for many unit testing frameworks, JUnit for example.
• The Ant build tool has a function to generate JUnit result xml files.
Health Report Amplification factor
• "Health report amplification factor" is a factor that multiplies the percent of
failures.
• Examples:
• Setting to "2" when the actual failure percentage is 1% results in a 98% health
score.
• Setting to "10" when the actual failure percentage is 3% results in a 70% health
score.
Procedure for Adding JUnit Reports to Projects
Freestyle (Traditional) Reports
From the Project Configuration Page
1. Click the "Add post-build action" dropdown.
2. Select "Publish JUnit test result report."
3. Set the relative path from the workspace to the result xml file.
Example from the course exercises:
//path
reports/result.xml
Ant Buid File Configuration Snippet to generate the test report:
32 -
5. Specify the path where the Cobertura report is generated.
6. Click Advanced; Configure the "Coverage Metric Targets" to reflect your goals.
• Metric targets can be set for Sunny/Stormy/Unstable for the following metrics:
• Methods
• Lines
• Conditionals
• Packages
• Files
• Classes
Notifications
• Important for reenforcing CI/CD ideals
• Prioritizing fixing the build when it fails is a central tenant of CI.
• Ensure contributors are aware of problems in a fast, distributed and automated
fashion.
• Configured in the post build action section of configuration of a project
Alarming
33 -
Jenkins notifications can be enhanced with post-build actions in project
configuration.
Failure Notifications
• If the build breaks, there should be a notification.
• At that point, the failure is expected without action, so Jenkins will not continue to
send
notifications.
• This is important to prevent "Alert Fatigue."
Integrations
• Email
• Set a static recipient for alerts.
• Editable Email Notification
• Email alerts with more options for notification content
• Slack
• HipChat
• Jabber
• IRC
Pipelines
34 -
2. Ensure there are no docker packages on the system already.
yum remove docker docker-common container-selinux
3. Ensure "yum-utils" is installed.
yum install -y yum-utils
4. Add the docker repo.
yum-config-manager --add-repo https://download.docker.com/linux/centos/
docker-ce.repo
5. Clear the Yum Cache.
yum clean all
6. Install the docker-ce package.
yum install docker-ce-17.03.0.ce-1.el7.centos
7. Add the jenkins user to the docker group.
gpasswd -a jenkins docker
8. Now, start the docker service.
systemctl start docker
9. Enable the docker service so it starts at system start time.
systemctl enable docker
10. Refresh the Jenkins service.
systemctl restart jenkins
Ant
• A command line tool for compiling, building and testing java projects
35 -
• Unlike Maven, it doesn't require project layout conventions.
Ant Installation
• You'll want to install Ant on all nodes that may build via an Ant file.
• If you're following a long with the course, you need to install Ant on the master
and the slave node.
1. Switch to the root user.
sudo su
2. Download Ant.
wget http://www.us.apache.org/dist/ant/binaries/apache-ant-1.10.1-bin.
tar.gz
3. Unpack the tar file.
tar xvfvz apache-ant-1.10.1-bin.tar.gz -C /opt
4. Symlink /opt/apache-ant-1.10.1 to /opt/ant
36 -
<property environment="env"/>
<!Sets variables which can later be used. ->
<!The value of a property is accessed via ${} ->
<property name="src.dir" location="src" />
<property name="build.dir" location="bin" />
<property name="dist.dir" location="dist" />
<path id="class.path">
<pathelement location="./lib/junit-4.10.jar" />
<pathelement location="./bin" />
</path>
<!Deletes the existing build, docs and dist directory->
<target name="clean">
<delete dir="${build.dir}" />
<delete dir="${dist.dir}" />
</target>
<!Creates the build, docs and dist directory->
<target name="makedir">
<mkdir dir="${build.dir}" />
<mkdir dir="${dist.dir}" />
</target>
<!Compiles the java code (including the usage of library for JUnit
->
<target name="compile" depends="clean, makedir">
<javac srcdir="${src.dir}" destdir="${build.dir}">
<classpath refid="class.path" />
</javac>
</target>
<!-Creates the deployable jar file ->
<target name="jar" depends="compile">
<jar destfile="${dist.dir}\rectangle.jar" basedir="${build.dir}">
<manifest>
<attribute name="Main-Class" value="Rectangulator" />
</manifest>
</jar>
37 -
<delete dir="${report.dir}" />
</target>
<target name="makedir" depends="clean">
<mkdir dir="${build.dir}" />
<mkdir dir="${report.dir}" />
</target>
<target name="compile" depends="makedir">
<javac srcdir="${src.dir}" destdir="${build.dir}">
<classpath refid="class.path" />
</javac>
</target>
<target name="test" depends="clean, makedir, compile">
<junit printsummary="yes" fork="true">
<classpath refid="class.path" />
<test name="RectangleTest" todir="${report.dir}" outfile="result">
<formatter type="xml" />
</test>
</junit>
</target>
<target name="main" depends="test">
<description>Main target</description>
</target>
</project>
The Jenkinsfile
38 -
echo 'Testing..'
}
}
stage('Deploy') {
steps {
echo 'Deploying....'
}
}
}
}
The Pipeline Directive
• It's required for the declarative format of the Jenkinsfile.
• It just says, "here's my pipeline definition."
pipeline {
.. .
}
The Agent Directive
• Where you specify which agent to run the pipeline, or a stage in the pipeline
• Can specify multiple agents for a pipeline
.. .
agent any
.. .
Agent Label Matching
• Expression matching works just like it would from the Project Configuration view.
// Matches our 'Slave 1' agent
agent {
label 'Slave 1'
}
39 -
• It will reach out to the Docker Hub if the container isn't available locally.
• It locates the container image based on the string you pass the directive.
• You must have Docker configured on the agents/master that in order to utilize it
as an agent.
// utilized the openjdk:8u121-jre image from the Docker Hub
stage('Test on Debian') {
agent {
docker 'openjdk:8u121-jre'
}
steps {
sh "wget http://brandon4232.mylabserver.com/rectangles/all/
rectangle_${env.BUILD_NUMBER}.jar"
sh "java -jar rectangle_${env.BUILD_NUMBER}.jar 2 3"
}
}
Agent Any and None
• If you declare an agent in the stage scope, you must declare "agent none" at the
top scope.
// Used when agents are defined in the "stage" scope
.. .
agent none
stages {
stage('Unit Testing') {
agent {
label 'Slave 1'
}
}
}
.. .
// Use "any" when any agent will work
.. .
agent any
stages {
stage('SO Simple') {
steps {
sh 'echo "I can run on any agent, except for Windows ;-)"'
40 -
• Deploy
• Stages are not at all limited by that scope though.
stages {
// 'Build' is simply setting the name of the stage
stage('Build') {
.. .
}
stage('Test') {
.. .
}
stage('Deploy') {
.. .
}
}
The Steps Directive
• You define the actual steps in the stage here.
• This would be analogous to a "Build Step" in the classic Project Configuration
view.
steps {
sh 'printenv'
sh 'ant -f test.xml -v'
junit "reports/${env.BUILD_NUMBER}_result.xml"
}
Common "Steps"
• sh: runs a shell script
• echo: prints a string
• junit: processes a junit report
• There are many more associated with various plugins.
The Post Directive
• Contains "Post-build" steps
post {
// This will always run, whether or not the previous steps
succeeded
always {
// Archiving the jar file, tracking fingerprint
archiveArtifacts artifacts: 'dist/*.jar', fingerprint: true
}
}
42 -
.. .
// Using the pollSCM trigger
triggers {
pollSCM('H 4/* 0 0 1-5')
}
The Parameters Directive
43 -
3. Create a file in the "vars" directory called "sayHello.groovy."
4. Populate the file with the following:
// vars/sayHello.groovy
def call(String name = 'human') {
// Any valid steps can be called from this code, just like in other
// Scripted Pipeline
echo "Hello, ${name}."
}
5. Commit and Push to your origin.
6. Go to the Jenkins Dashboard.
7. Click "Manage Jenkins," then "Configure System."
8. Go to the "Global Pipeline" and click "Add."
9. Add a Name for the library, like "My Pipeline Library."
10. Set "master" for the "Default version." This can be any Git ref available from
the origin (branch, tag,
commit, etc...).
44 -
}
Console Output
.. .
Recording test results
[Pipeline] echo
Hello, Pinehead.
[Pipeline] }
.. .
Adding Your First Pipeline
Prerequisites
• You have a Git repository with a project you'd like to set up for a Pipeline.
• You have a Jenkins setup and configured per the steps in the video/guide.
• Appropriate keys are set up for GitHub.
• Use the below example if you just want to learn the process:
Sample Repo Setup Procedure
1. Create a new repo on GitHub.
• Sign up for a new account if necessary.
• Call the repo "first-jenkins-pipeline."
2. Clone to your dev environment.
3. From the terminal, set the git global config items if necessary:
• git config --global user.name "Your Name"
• git config --global user.email you@example.com
4. Create a Jenkinsfile in the root path.
• vi Jenkinsfile
• Add the following in a Jenkinsfile in the root path of the git repo:
pipeline {
agent { docker 'ruby' }
stages {
stage('build') {
steps {
46 -
• Using REST API to trigger jobs remotely, access job status, create/delete/copy
jobs
• There are Ruby/Python/XML/JSON APIs.
Disabling "Prevent Cross Site Request Forgery exploits"
• You man have trouble authenticating via the REST api for some commands with
some versions of
Jenkins.
• Disabling "Prevent Cross Site Request Forgery exploits" can resolve this.
• Uncheck the box from "Manage Jenkins" -> "Configure Global Security"
Generating a Token
• You can generate a crumb to authenticate with the Jenkins server
• You should use this, and have $CRUMB available via Environment Variables,
before attempting the
other commands below.
CRUMB=$(curl -s 'http://USER:PASS@<your-server>:8080/crumbIssuer/api/
xml?xpath=concat(//crumbRequestField,":",//crumb)')
47 -
• HTTP POST on 'JENKINS_URL/job/JOBNAME/build?token=TOKEN'
• '%20' for spaces in the URL
• Example:
curl -X POST -H "$CRUMB" http://<your-server>.co m:8080/job/My%20
Freestyle%20Project/build \
+-data token=build \
+-data-urlencode json='{"parameter": [{"name":"branch",
"value":"development"}]}'
JSON Format
{
"parameter": [
{
"name":"first_param",
"value":"first_value"
},
{
"name":"second_param",
"value":"second_value"
}
]
}
Other Common API Calls
Retrieve or Update the Config
• You can retrieve and update the project configuration via XML or JSON.
• You can programmatically update the xml and "PUT" it back to change config.
//xml format
curl -H "$CRUMB" http://<your-server>.co m:8080/job/Freestyles/job/
My%20Freestyle%20Project/config.xml
//json format
curl -H "$CRUMB" http://<your-server>.co m:8080/job/Freestyles/job/
My%20Freestyle%20Project/api/json
config.xml
<?xml version='1.0' encoding='UTF-8'?>
48 -
plugin="github@1.26.0">
<projectUrl>https://github.com/labmac/jenkins-test/</projectUrl>
<displayName></displayName>
</com.coravy.hudson.plugins.github.GithubProjectProperty>
<hudson.model.ParametersDefinitionProperty>
<parameterDefinitions>
<hudson.model.S tringParameterDefinition>
<name>BRANCH</name>
<description></description>
<defaultValue>master</defaultValue>
</hudson.model.S tringParameterDefinition>
</parameterDefinitions>
</hudson.model.ParametersDefinitionProperty>
</properties>
<scm class="hudson.plugins.git.GitSCM" plugin="git@3.0.5">
<configVersion>2</configVersion>
<userRemoteConfigs>
<hudson.plugins.git.UserRemoteConfig>
<url>git@github.com:labmac/jenkins-test.git</url>
<credentialsId>redacted</credentialsId>
</hudson.plugins.git.UserRemoteConfig>
</userRemoteConfigs>
<branches>
<hudson.plugins.git.BranchSpec>
<name>*/$BRANCH</name>
</hudson.plugins.git.BranchSpec>
</branches>
<doGenerateSubmoduleConfigurations>false</
doGenerateSubmoduleConfigurations>
<browser class="hudson.plugins.git.browser.GithubWeb">
<url>https://github.com/labmac/jenkins-test/</url>
</browser>
<submoduleCfg class="list"/>
<extensions/>
</scm>
<assignedNode>Linux && CentOS</assignedNode>
<canRoam>false</canRoam>
<disabled>false</disabled>
<blockBuildWhenDownstreamBuilding>false</
blockBuildWhenDownstreamBuilding>
49 -
<configs>
<hudson.plugins.parameterizedtrigger.
PredefinedBuildParameters>
<properties>IMPORTANT_PARAMETER=$BUILD_NUMBER</properties>
</hudson.plugins.parameterizedtrigger.
PredefinedBuildParameters>
</configs>
<projects>Downstream Project</projects>
<condition>ALWAYS</condition>
<triggerWithNoParameters>false</triggerWithNoParameters>
<triggerFromChildProjects>false</triggerFromChildProjects>
<buildAllNodesWithLabel>false</buildAllNodesWithLabel>
</hudson.plugins.parameterizedtrigger.
BlockableBuildTriggerConfig>
</configs>
</hudson.plugins.parameterizedtrigger.TriggerBuilder>
<hudson.tasks.Shell>
<command>docker run hello-world</command>
</hudson.tasks.Shell>
<hudson.plugins.copyartifact.CopyArtifact
plugin="copyartifact@1.38.1">
<project>My Java Project</project>
<filter>dist/rectangle.jar</filter>
<target></target>
<excludes></excludes>
<selector class="hudson.plugins.copyartifact.
StatusBuildSelector"/>
<doNotFingerprintArtifacts>false</doNotFingerprintArtifacts>
</hudson.plugins.copyartifact.CopyArtifact>
<hudson.tasks.Shell>
50 -
Freestyle%20Project/disable
Update description
curl -H "$CRUMB" http://<your-server>.co m:8080/job/Freestyles/job/My%20
Freestyle%20Project/description
Jenkins CLI
Set Up a Pub Key for Your User
1. Generate a rsa key pair for your user.
2. Go to: http://yourserver.com/me/configure
3. Copy the contents of your id_rsa.pub file into the "SSH Public Keys" field.
Download the CLI Jar file
wget -P /var/lib/jenkins/ http://<your-server>:8080/jnlpJars/jenkinscli.
jar
Executing Commands
java -jar /var/lib/jenkins/jenkins-cli.jar -s http://<your-server>:8080/
<command>
(Optional) Setting Up Your Environment
• You can use an alias and environment variables to make the syntax a bit simpler.
• The instructions below assume you're running from the Jenkins Master and from a
CentOS system.
• Make changes as necessary if that's not the case.
1. Set the JENKINS_URL environment variable.
echo "JENKINS_URL='http://localhost:8080'" >> /etc/environment
2. Set an alias for your bash shell.
echo "alias jenkins-cli='java -jar /var/lib/jenkins/jenkins-cli.jar'" >>
51 -
~/.bashrc
3. The Jenkins CLI can now be executed via jenkins-cli
Common Commands
// Returns user
52 -
// Returns console output from a build
[root@brandon4231 ~]# jenkins-cli console "Freestyles/My Freestyle
Project" 51
Started from command line by
ha:AAAAmR+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyIL
UoJz
Mv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinB
IGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAz2EgZe/dLi1CL9pKLEvJT8P
ACqY1WJwgAAAA==brandon
Building remotely on ha:AAAAnR+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vO
D8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk
8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAz2Eg
ZB/
eT83ILSktQi/eCcxLJUBUMALBjAR8YAAAA=Slave 1 (Linux CentOS) in workspace /
var/lib/jenkins/workspace/Freestyles/My Freestyle Project
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
.. .
53 -
for (slave in hudson.model.Hudson.instance.slaves)
{
FilePath fp = slave.createPath(slave.getRootPath().toString() +
File.separator + "workspace");
fp.deleteRecursive();
• Plugin Printer
• https://github.com/jenkinsci/jenkins-
scripts/blob/master/scriptler/pluglist_print.groovy
/*** BEGIN META {
"name" : "plugins lister",
"comment" : "print list of plugins (optionally set build.
displayName)",
"parameters" : [],
"core": "0.601",
"authors" : [
{ name : "Mark Hudson" }
]
} END META**/
def pcount = 0 ; def pstr = ''
def plist = jenkins.model.Jenkins.instance.pluginManager.plugins
plist.sort{it} // plugins list it defaults to shortName
plist.each {
pcount = pcount + 1
pname = (pcount + ' ' + it).replaceAll("Plugin:", '')
54 -
to ensure that build numbers do not get reset to 1 when migrating jobs
from one Jenkins server to another.",
"parameters" : ['rootItem'],
"core": "1.625",
"authors" : [
{ name : "Amit Modak" }
]
} END META**/
import jenkins.model.*
def getBuildNumber(def ite m, def node) {
if(item instanceof com.cloudbees.hudson.plugins.folder.Folder) {
node[ite m.getName()] = [:]
item.getItems().each {
getBuildNumber(it, node[ite m.getName()])
}
} else {
node[ite m.getName()] = ite m.nextBuildNumber
}
}
//
// main