Introduction

This page provides information on setting up a basic Jenkins environment (running on Windows) which will automatically sync from a Perforce server (running on Linux), build binaries, and submit binaries back to Perforce. This page assumes that you:

• Have no experience with Jenkins,

• Have a running Perforce server on a Linux VPS (some steps may need to be altered for a Windows Perforce server),

• Are experienced in Perforce and Linux usage, and

• Are using an EGS Launcher version of the Unreal Editor (some steps may need to be altered or added to use a source-build engine).

Goal

This example setup uses stream depots with Perforce (it should be possible to use classic depots if preferred, you will need to determine suitable alternatives for some steps). The intended workflow is for general users to work in the regular mainline stream and for coders will work in a virtual stream of mainline which excludes all Binaries folders in your project. Jenkins will work in a virtual stream of mainline which excludes any Content folders. The result is that coders can "just work", creating code changes and submitting them. Jenkins will be notified by Perforce when a source code file is changed in the mainline stream. Jenkins will sync to the latest changes from Perforce (ignoring any Content folders to speed things up), initiate a build, and add generated DLLs to a new Changelist and submit it. Artists/designers may then receive the DLLs the next time they run Get Latest.

Prerequisites

1. The machine running Jenkins must be capable of syncing from Perforce and building Unreal projects. This means the machine must:

   • Have Perforce installed,

   • Have Visual Studio Build Tools installed (you can install Visual Studio Community Edition, or you can install the basic CLI Build Tools), and

   • Have Unreal Engine/Editor installed (or plan to build it from source, possibly with a separate Jenkins Job, this is beyond the scope of this example),

2. This setup may need to be used on a computer which is not actively used for UE4 development. Live Coding may prevent MSVS builds from running in the background by Jenkins while the editor is open, even if you set up Jenkins on a separate user login (DRAFT NOTE: unsure if this is still true). It may also be possible to install and run everything within a Virtual Machine (untested, and beyond the scope of this example).

Contents

Perforce Setup

The author recommends using stream depots. Recommended reading for newcomers:

• Perforce documentation on Streams Depots

• (Old) Perforce documentation containing useful explanation of stream mapping keywords

Jenkins Stream

Create a virtual stream based off of your project's Mainline stream named ~Jenkins (the tilde in this example does nothing, it is simply a marker to help indicate that this stream is not intended for human use). Then, in the Advanced tab of the Stream settings, under Paths, enter:

share Binaries/...     (Share entries include the mapped folder into the stream for regular read/write use)
share Plugins/SomePluginName/Binaries/...      (Specify any extra Plugins' Binaries folders too)
exclude Content/...     (Exclude entries prevent the stream from syncing/downloading the specified folder)
exclude ContentSource/...      (Specify any other unnecessary Content/Art/Other folders too)
isolate ...      (Isolate entries include the mapped folder for read-only use, this will include the Source folder)

Coders Stream

Create a second virtual stream also based off of the Mainline stream named MainlineCoding. In the Advanced tab, under Paths, enter:

share ...
exclude Binaries/...
exclude Plugins/SomePluginName/Binaries/...    (Specify any Plugin Binaries folders like this)

Once done, you should have a Stream Graph that looks something like this.

Workspaces

Ensure general developers create their workspace using the Mainline stream and coders create their workspace using the MainlineCoding stream.

Jenkins - Installing

Download and install from Jenkins Website.

Jenkins may prompt you whether to run as LocalSystem or under a User Account. It is recommended to run Jenkins under a User Account. LocalSystem is considered an insecure global user and you may run into permissions issues in the future.

When setup completes, Jenkins should automatically launch. After following the instructions to Unlock Jenkins, it will prompt you about plugins. Choose Select plugins to install. Ensure the following plugins are selected (others are optional). We will add more plugins later from within Jenkins.

• P4

• Matrix Authorization Strategy

When prompted, specify login info for an Admin User.

When prompted, specify the URL from which Jenkins will be accessible by all users. This should be something like http://JenkinsServerPublicIPAddress:8080/. Jenkins may insert this URL into page data so it may be necessary to avoid users from getting broken links.

Installation should now be complete. Jenkins will open up to its front page.

Jenkins - Creating a build user and enabling anonymous access

We will create a separate Jenkins "build user" which only has permissions to control builds, and we will enable public read-only access which is useful for allowing team members to look at build logs.

Create a build user

From the main page, select Manage Jenkins. Select Manage Users. Select Create User. Enter in user data and press OK. You are returned to the Users page. Below I've created user buildbot.

Set up security permissions

Go to Manage Jenkins. Select Configure Global Security. Under Authorization, choose Matrix-based security. Press Add user or group... and enter your build user's name. Set up your security matrix as desired, see example below.

Create an API Token for your build user

A token will be used later to authenticate remote requests. Open a second browser window in incognito/private mode. Log in to Jenkins again (e.g. go to localhost:8080), this time logging in as the build user you created. Click on your username on the top bar to bring you to the user page. Click on Configure. Under API Token press Add new token. Give it a name (e.g. perforce) and press Generate. Copy this token somewhere safe for now. Press Save and close this browser window.

Jenkins - Getting necessary plugins

This example setup will use the following additional plugins:

• PostBuildScript Plugin

• Pre SCM BuildStep Plugin

• SCM Skip Plugin

• Environment Injector Plugin

Go to Manage Jenkins. Go to Manage Plugins. Click on the Available tab. Search for and install each of the above plugins (search for a plugin and check it on, then search for the next one and check it on), press Download now and install after restart. After the screen changes, select Restart Jenkins when installation is complete and no jobs are running. Wait for it to complete, and log into Jenkins again.

Jenkins - Setting up global settings

Environment Variables

Go to Manage Jenkins. Select Configure System. Under Global Properties turn on Environment Variables. Add a variable PROJECTNAME_UE4_REBUILDBAT. Our Job will use this variable to run the UE4 build batch file.

Set up Perforce Credentials

Go to Manage Jenkins. Select Manage Credentials. Click into the Global store (beside the Jenkins domain). Select Add Credentials from the left pane. Change the Kind to a Perforce Password Credential and specify your desired Perforce login info, which Jenkins will use to connect to Perforce.

Jenkins - Creating the Job

At the front page, select New Item from the left pane. Select Freestyle project, give the Job a name, and press OK. The configuration window for a new Job appears.

Source Code Management

Select Perforce Software'. Beside Perforce Credentials, select the credential you created before. Under Workspace Behaviour, select Streams and type your stream name into the Stream Codeline (e.g.//StreamDepotName/MainlineStream). Under Populate Options, select Auto cleanup and sync.

Build Triggers

You can leave everything here unchecked. For information only, temporarily check on Trigger builds remotely (e.g., from scripts). Note that it states we can use a URL to trigger builds remotely - take note of this URL, we will use it later. Jenkins does not actually need this option turned on for remote triggered builds to work, unless you want to use an Authentication Token. We will skip use of a Token for this example; uncheck Trigger builds remotely.

Build Environment

Select Inject environment variables to the build process. Under Properties Content add:

Build

Click Add build step and select Execute Windows batch command. Enter:

CALL "%PROJECTNAME_UE4_REBUILDBAT%" "%BUILDCONFIG%" "%PLATFORM%" "%CONFIG%" "%WORKSPACE%\%PROJECTNAME%.uproject" -WaitMutex -FromMsBuild
IF %ERRORLEVEL% NEQ 0 (EXIT %ERRORLEVEL%)

Post-build Actions

Click Add post-build action and select Perforce: Publish assets. Set it up similarly as the Source Code Management section above. Under Publish options, select Submit change. Enter a suitable description (e.g. "Binaries updated by Jenkins. Build: ${BUILD_TAG}"). Activate Only publish on build success. Click on Advanced, and in Publish paths enter any folders in the depot which will contain binary files that are being updated. This should include your main game project Binaries folder as well as any Plugins\PluginName\Binaries folders.

Press Save. You will be returned to the front page for the Job.

Test it

At this point we should have a working Job. Press Build Now from the left pane. A build should appear running under the Build History. Click on the dropdown arrow beside the build entry and click on Console Output to watch the build progress.

Perforce - Creating a Build-trigger Script

To trigger Jenkins to run a build, we will use cURL ran from a shell script which Perforce executes. SSH into your Perforce server and navigate into your Perforce server directory (e.g. /opt/perforce). Create a new directory triggers. Create a new text file called execute_jenkins_p4trigger.sh. Write the following script:

#!/bin/bash
PROJECT=$1

JUSER=JenkinsBuildUserLogin
JTOKEN=BuildUserAPIToken
JSERVER=http://JenkinsServerAddress:Port

curl -X POST $JSERVER/job/$PROJECT/build --user $JUSER:$JTOKEN

Save the file and quit, and enable execution permission on the file (e.g. run the chmod command like chmod +x execute_jenkins_p4trigger.sh). Keep your SSH session open for the next step.

Perforce - Setting up Triggers to Launch the Build-trigger Script

On your Perforce server run the command p4 triggers. A text editor should open with your Perforce trigger settings. Add the following trigger entries below the Triggers: line:

        jenkins_build_projectname_mainline change-commit //DepotName/Mainline/....cpp "/opt/perforce/triggers/start_jenkins_build.sh JenkinsJobName"
        jenkins_build_projectname_mainline change-commit //DepotName/Mainline/....h "/opt/perforce/triggers/start_jenkins_build.sh JenkinsJobName"
        jenkins_build_projectname_mainline change-commit //DepotName/Mainline/....cs "/opt/perforce/triggers/start_jenkins_build.sh JenkinsJobName"

This adds three triggers, whenever a commit modifies a code file (.cpp, .h, .cs) in the specified streamname (e.g. //DepotName/Mainline/) Perforce will run the specified script with the specified Jenkins Job as an argument.

Note: the syntax of the triggers entries is important. It may be necessary for example to include some indentation at the start of each trigger. Comments are not allowed.