Skip to end of metadata
Go to start of metadata

Overview.

Jenkins is a cross-platform, continuous integration and continuous delivery application that increases your productivity.
Use Jenkins to build and test your software projects continuously making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build.
Key features include:

  • Plugins for many popular builders such as Ant, Maven, ...; source control such as CVS, SVN, Git, Mercurial, ...; integration with GitHub through various plugins; notification via email and and other means; authentication via default container authentication/OpenID/Custom users/...
  • Other 3rd part extensions for lots of other nice features (example SauceLabs plugin for easier browser testing in the cloud).
  • Multiple jobs can be made to run at the same time, even on multiple master/slave Jenkins nodes and, using some plugins this can be configured around a number of available external 'resources'.
  • Groovy scripts can be run directly from the web interface against a Jenkins API.
  • Open source so extensible and customizable.
  • Web application - the entire build/job configuration can be done via a web browser.

Installing Jenkins

Download and install Jenkins native package from https://jenkins.io/. Jenkins can also be deployed as a .war file in containers such as Tomcat, but the configuration would be identical.

For the purpose of this tutorial we will use the native Jenkins stand-alone installer.

You should now be able to access Jenkins at http://localhost:8080. This tutorial is based on a native Windows service installation, but it should be similar on other OSes as well.

Install JDK 1.8.x. http://www.oracle.com/technetwork/java/javase/downloads/index.html as well.

Jenkins Configuration Tips

As Jenkins and Servoy are both Java driven applications, they share some Java technologies, and by default, share port configurations as well. If you plan to run Jenkins and Servoy Application Server on the same machine, port changes will need to be made one application or the other in order for the applications to work properly.

Only one or the other need to be modified, not both.

Changing ports for a Servoy Application Server installation is detailed here.

To change port configurations for Jenkins instead, modify the jenkins.xml file found in the Jenkins main directory. See the section below:

jenkins.xml
  <arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -jar "%BASE%\jenkins.war" --httpPort=8080</arguments>

Note that '--httpPort=8080' is in conflict with Servoy Application Server default port. Change this port to any other available port on the machine (for example 8085) then restart the Jenkins service. The new URL to access Jenkins will be http://localhost:8085. (If you plan to install Jenkins as a .war inside another container such as tomcat, then that container's configuration determines the port and the URL will be something like http://localhost:8085/jenkins)

Jenkins is fully configurable from the browser. It has many useful plugins. If you think you need something, look through the available plugins. What you need is probably already available.

In this tutorial we will use some plugins (to install them go - in the browser - to Manage Jenkins > Manage plugins):

  • Google Login plugin (allows authentication based on google account (or company Google account)). If you do not want to use google login you can use other login plugins or even make your own custom auth. and users in Jenkins
  • Build timeout plugin (makes sure jobs will automatically be stopped after a timeout - so your build fails if it take too long for some reason, instead of just never ending)
  • Git plugin: for access to the sample Git repositories; you will need to install a git client as well: https://git-scm.com/download
  • Multiple SCMs plugin: allows us to nicely clone multiple Git repositories (Git plugin doesn't allow that directly (multiple each in it's own subfolder) right now)

Some other plugins that we will use are pre-installed.

Configure Security Using Google Login plugin (skip this step if you are not interested in authentication on the Jenkins server)

  • see "Google Login Plugin" for how the plugin can be configured.
  • go to Manage Jenkins > Configure global security
  • check Enable security
  • choose Matrix-based security
  • type your google username in User/group to add and click on "Add button
  • grant yourself all rights by checking all checkboxes in your user's row
  • uncheck all checkboxes in the Anonymous row, except for Job - Discover (this one allows people to get prompted with the login when try to access a job's URL). Please do this only after you granted yourself all rights, otherwise noone will be able to access Jenkins.
  • click on Save

Now you should be the only one that can access the Jenkins server using your google credentials.

You can add any other people and configure their rights as needed now.

Note: Jenkins access can be configured in other ways as well, if you don't have a google/other openID account or you just want to use something else (LDAP/custom users).

Other Things to Configure

Here are some settings you should/could tweak in Manage JenkinsConfigure System:

  • add a JDK and point it to the one you just installed (uncheck 'install automatically').
  • if you don't have Ant installed, in Ant section add a new installation, give it a name and check Install automatically.
  • '# of executors': set it to 1 for now; this limits the concurrent executing jobs to 1. In the future if you want to set-up a more advanced configuration you can configure multiple jobs to run simultaneously (for example for multiple solutions) - just make sure they use separate resources so that they don't affect each other (for example separate Servoy Installations).
  • 'Jenkins URL': to the real URL that will be used by users to access the Jenkins server. (for example this will appear as part of notification emails)
  • 'System Admin e-mail address': use your email address.
  • 'E-mail Notification': configure this as needed. By default it will try to use the localhost SMTP server which will already be working probably on some linux machines. You can also test the configuration by sending a test email.
  • hit the Save button.

Step-by-step setup: Sample Servoy Software Factory on Jenkins

The sample Servoy Software Factory setup uses 3 Git repositories:

Running one or all steps

The build scripts above are build in such a way that you can enable only the build steps you are interested in:

  1. run (smart client) unit tests
  2. run mobile client unit tests
  3. run end-to-end tests
  4. deploy solutions to the manual test server

So you can enable any combination of steps: (1) or (1, 4) or (4) or (1, 2, 3 ,4) or ... . More about how you can do this will be described later. You can create one Jenkins job as in the example below or you could define separate jobs for each step.

This is what will happen when a Servoy - Jenkins job (will be created later in this chapter) runs:

  1. changes are fetched from Git (solution projects, resources project(s), build scripts, tools);
  2. (if that build step is enabled) solutions to be (smart client) unit tested are exported as .servoy
  3. (if that build step is enabled) solutions to be unit tested in mobile client are exported as .servoy (the attached test service solution) and .war
  4. (if that build step is enabled) solutions to be end-to-end tested are exported as .war files (Servoy war deployment including the solution)
  5. (if that build step is enabled) solutions to be deployed to the manual test server are exported as .war files (Servoy war deployment including the solution)
  6. (if that build step is enabled) unit test (smart client) client starts a database repository - based on the repository database that must be defined in the according servoy.properties (if it is not defined to be an in-memory server / HSQL, then the database must be online; for the sample workspace it will use an in-memory server, so no database needs to be started). Solutions from (2) are imported into that repository (databases needed by the solutions must be online if not defined as in-memory; for the sample workspace no database server needs to be started) and unit tests run in a slightly modified smart client - to not block with modal dialogs when errors occur for example.
  7. (if that build step is enabled) mobile unit test client starts a database repository - based on the repository database that must be defined in the according servoy.properties (if it is not defined to be an in-memory server / HSQL, then the database must be online; for the sample workspace it will use an in-memory server, so no database needs to be started). Solutions from (3) are imported into that repository or (for mobile .war files) in Servoy's embedded Tomcat (databases needed by the solutions must be online if not defined as in-memory; for the sample workspace no database server needs to be started) and unit tests run in a slightly modified version of mobile client.
  8. (if that build step is enabled) the Tomcat server from software_factory_build_tools repository is started. End-to-end .war exports from (4) are each deployed on that tomcat, protractor and/or selenese runnner end-to-end ngclient tests run against it and then it gets undeployed. After all end-to-end tests are done the Tomcat server is shut down.
  9. (if that build step is enabled) Deployment .war exports from (5) are deployed to an already running manual testing Tomcat server.
  10. test results and log files are archived by the Jenkins job for further reference; emails are sent as needed.

 

Servoy Exporters and Database connections

Because the Servoy Workspace Exporters are configured by the build scripts above to use the '-dbi' option, they will not need database connections to be online when exporting, but the servoy.properties must still contain the required server definitions; for the sample repository, the servoy.properties file is already configured correctly.

Step 1 -- Install Servoy Developer

Install Servoy Developer 8.x. It will be used on the machine running Jenkins for exporting test solutions and running tests.

It's best to use a clean installation (no extra eclipse plugins) to avoid potential problems generated by other eclipse plugins (for example installing Subclipse in this developer can cause a hang as that plugin is trying to access the UI while blocking a resource - in the headless solution export environment).

Step 2 -- Configure the Jenkins Job

Navigate to the Jenkins server page in your browser. We will set up what is common/needed to all steps below, then for each step - if you want to enable it - we will do what is needed.

Click on New Item > Free-style project, and set the Job name to 'test_and_deploy' ('_' is used to avoid problems with spaces in paths where used as command line arguments - extra care is needed otherwise).

Under 'Advanced Project Options' click the button then in 'Display Name' type 'Export, test and deploy sample Servoy solutions'.

Under 'Source Code Management' select 'Multiple SCMs'. We will add the repositories needed for the job one by one:

Normally you would fork the software_factory_sample_solutions repository on github and use that fork at point 2 above - so that you can play with it - change and push changes to Git server.

Then under 'Build Triggers' select 'Poll SCM' and in 'Schedule' field use H/3 * * * * (every 3 minutes, let Jenkins choose the minute when it checks for Git changes - to avoid overcrowding in case you will define multiple jobs in the future). You can increase that to decrease requests on the Git (github in this case) server. If changes are found Jenkins will trigger the build. If you want a check every minute use * * * * *. But for now you can use the sample repository directly as suggested above just to see it running.

Poll SCM vs triggers

It is always better to configure Git triggers instead of polling. For example - when using GitHub , you have GitHub plugin(s) that allow that in Jenkins (+corresponding configuration on GitHub repositories to call/trigger the Jenkins job when changes get pushed). That means that instead of Jenkins checking for changes now and then you make GitHub ask Jenkins to start a job right away when changes get pushed to the repository; it starts jobs faster and it also removes poll network load.

Under 'Build Environment' check 'Abort the build if it's stuck', then select 'Absolute' - 30 minutes. This will make sure that the build ends with failure if something gets stuck (if the build runs much longer then usual). This is optional and of course you might need to tweak this depending on how long your tests usually take and what they do (you could also try other configurations, for example 'Elastic' - 150% - 3 builds - 30 minutes). This option is contributed to Jenkins by the 'build timeout plugin'.

Under 'Build' click on Add build step. Choose Invoke Ant. Select the version you configured to auto-install at 'Other things to configure' above. Click on Advanced. In 'Targets' type 'main'. In 'Build file' type software_factory_build_scripts/servoyAntScripts/jenkins_build.xml. In 'Properties', expand the field (arrow down button) then type:

servoy.install.dir        = [path_to_your_separate_Servoy_developer_installation_from_Step_1]
workspace.solutions.dir   = ../../software_factory_sample_solutions

If you are on windows use '/' or '\\' instead of '\' in that path. For example servoy.install.dir=f:/ServoyInstalls/Servoy_8.

Under 'Post-build Actions':

  • add 'Archive the artifacts' with this in 'Files to archive': software_factory_build_scripts/buildResults/exports/logs/**/*.txt; click on 'Advanced' and check 'Do not fail build if archiving returns nothing'.
  • add 'E-mail notification' and type your email address in 'Recipients' field.

Click on Save and go back to Jenkins dashboard. The new job you just created 'Export, test and deploy sample Servoy solutions' should automatically start in under 3 minutes (or you can start it manually using the play icon next to it). It should pass, although it's not doing anything useful right now - except getting the code from the Git repositories.

If anything goes wrong you can click on your job's name then on the left side you see a list of builds. Click on the build that failed and you can see it's details (time/status/tests/console/...).

I also like to have 'AUTO REFRESH' enabled as well (a link in Jenkins dashboard) - just to make sure the page is showing the latest status of Jenkins.

Step 3 -- Configure/enable (smart client) Unit Tests

If you wish the Jenkins job configured above to run unit tests on sample 'solution1' and 'solution2' you can enable that by going to the job configuration page and:

  • in 'Invoke Ant -> Advanced' expand the 'Properties'. Add the following lines after what is already there:

    build.run.unit.tests                = yep
    smart_test_solutions                = solution1,solution2
  • in 'Archive the artifacts' add , software_factory_build_scripts/buildResults/jsunit_results/logs/**/*.txt, software_factory_build_scripts/buildResults/jsunit_results/junit-noframes.html
  • in 'Post-build Actions' click on 'Add post-build action' and choose 'Publish JUnit test result report'; check 'Retain long standard output/error' and in 'Test report XMLs' put software_factory_build_scripts/buildResults/jsunit_results/TESTS-TestSuites.xml

About that 'yep'

Wherever you see in these properties '= yep' the actual string value of 'yep' is irrelevant. It is just put there to define that property. Checks will be made in the ant scripts on whether or not that property is defined, not on the value it has. So if you want to negate/disable that property just comment it out or don't define it. Because if you define it with any other value (e.g. 'yes', 'true', 'false', 'no', 'nope', 'anything') it means the same thing as 'yep' - it means the property is defined. For example if you do not want unit tests to run you should remove completely 'build.run.unit.tests = yep' or comment it out '#build.run.unit.tests = yep'. Do not try to set it to 'build.run.unit.tests = nope' because that won't work.

 

Click 'Save'. Now run the job again by pressing 'Build Now'. It should run unit tests for the two solutions and finish with success.

The job will also generate some artifacts. Go to the job, then the last build number, status page. Under 'Build Artifacts' you should see log files for exporter, smart import test client as well as an html with test results. Below that there's also a 'Test Result (no failures)' you can click on - that is another way of checking out the test results.

The pre & post import hooks of solutions that are unit-tested are triggered when the .servoy files get imported. So you can take advantage of that if you need to set-up the test environment.

Running unit tests on HEADLESS Linux machines

If you are setting this up on a headless Linux, make sure to have an X11 (UI) installed (required by the Servoy import unit test client).

Xvfb can be used for Linux machines that are supposed to be headless.

It can be installed either with apt-get: sudo apt-get install xvfb

or with yum: yum install xorg-x11-server-Xvfb

Xvfb should be run from a Jenkins job by using the plugin  https://wiki.jenkins-ci.org/display/JENKINS/Xvfb+Plugin .

Step 4 -- Configure/enable mobile client Unit Tests

If you wish the Jenkins job configured above to run mobile client unit tests on sample solution 'mobile_ipAddressValidatorTester' you can enable that as shown below. If not you can skip to the next step.

For mobile testing we will need Chrome browser installed (other browsers, even mobile device simulations/emulations or cloud test services can be used, check the help page of Servoy mobile test runner and advanced customization for details), a copy of SeleniumServer jar downloaded (http://docs.seleniumhq.org/download/) and a copy of Chrome driver downloaded/extracted (https://sites.google.com/a/chromium.org/chromedriver/). The mobile test solution will automatically be deployed in Servoy Developer's bundled Tomcat server (the one at [servoy.install.dir]) and started using Selenium in the Chrome browser to run tests - by the mobile servoy test suite. If the tests pass, the actual mobile solution will be exported for deployment and copied to/deployed in [mobile.deploy.webapps.dir]'s bundled Tomcat, if that property is set.

To enable mobile client unit testing, go to the configuration page of the job created above and:

  • go to 'Invoke Ant -> Advanced', expand 'Properties' and add to the already existing ones (replace the content between brackets with the locations you have downloaded the tools mentioned above to; make sure the servoy.install.url (where the Servoy that you just installed is accessible) does not conflict with any Servoy instance you might already have running):

 

build.run.mobile.unit.tests         = yep
servoy.install.url                  = http://localhost:8080
webdriver.chrome.driver             = [Path_to_chromedriver_downloaded_above._Eg:_c:/tools/chromedriver.exe]
selenium.server.jar                 = [Path_to_selenium_server_jar_downloaded_above._Eg:_c:/tools/selenium-server-standalone-3.141.59.jar]
mobile_test_solutions               = mobile_ipAddressValidatorTester
mobile_service_solutions            = mobile_ipAddressValidatorTester_service
mobile_to_deploy                    = mobile_ipAddressValidator
mobile_to_deploy_services           = mobile_ipAddressValidator_service
#mobile.deploy.webapps.dir          = [uncomment_and_set_value_to_where_a_successful_mobile_war_will_be_copied_for_deploy_if_you_want_to_also_deploy_the_mobile_war_and_set_value._Eg:_${mobile.deploy.servoy.install.dir}/application_server/server/webapps]
  • (if you didn't already do this at Step 3) in 'Archive the artifacts' add , software_factory_build_scripts/buildResults/jsunit_results/logs/**/*.txt, software_factory_build_scripts/buildResults/jsunit_results/junit-noframes.html
  • (if you didn't already do this at Step 3) in 'Post-build Actions' click on 'Add post-build action' and choose 'Publish JUnit test result report'; check 'Retain long standard output/error' and in 'Test report XMLs' put software_factory_build_scripts/buildResults/jsunit_results/TESTS-TestSuites.xml

Click 'Save'. Now run the job again from the 'play button'. It should also run the mobile unit tests and finish with success.

The job will generate some more artifacts. Go to the job, then the last build number, status page. Under 'Build Artifacts' you should see log files for exporter, mobile test client as well as an html with test results. Below that there's also a 'Test Result (no failures)' you can click on - that is another way of checking out the test results.

The headless Linux note from above applies to mobile unit tests as well.

Step 5 -- Configure/enable ng-client End-to-End Tests

If you wish the Jenkins job configured above to run end-to-end tests on sample 'solution1' you can enable that as shown below.

Install 'npm' and make that command available in path - a nodejs install https://nodejs.org/ will bring npm with it.

Go to the job configuration page and:

  • click 'Add SCM -> Git'. fill 'Repository URL' with https://github.com/Servoy/software_factory_build_tools.git ; in 'Additional Behariors' choose 'Add -> Check out to a subdirectory' and type in 'Local subdirectory for repo' field the value software_factory_build_tools.
  • in 'Invoke Ant -> Advanced' expand the 'Properties'. Add the following lines after what is already there (replace the content between brackets):

    build.run.e2e.tests                 = yep
    ngclient_e2e_test_solutions         = solution1
    e2e.tomcat.start_stop.tools.tomcat  = yep
    e2e.tomcat.java.home                = [c:/Program Files/Java/jdk1.8.0_201]
  • in 'Archive the artifacts' add , software_factory_build_scripts/buildResults/e2e_results/logs/*.txt
  • in 'Post-build Actions' click on 'Add post-build action' and choose 'Publish JUnit test result report'; check 'Retain long standard output/error' and in 'Test report XMLs' add , software_factory_build_scripts/buildResults/e2e_results/*.xml

Click 'Save'. Now run the job again from the 'play button'. It should run end-to-end tests for solution1 and finish with success.

The job will also generate some more artifacts. Go to the job, then the last build number, status page. Under 'Build Artifacts' you should see log files for exporter and a file containing the Servoy app. server logs while running the end-to-end tests. Below that, test results are also added to 'Test Result (no failures)' - this time there is no html report generated separately.

The end-to-end testing step runs the protractor and selenese runner test scripts in software_factory_sample_solutions/build_files/e2e_test_scripts; protractor test scripts are in spec/[solution_name] and the selenium test scripts are in selenium/[solution_name]. The test script supports testing multiple solutions sequentially if needed, just provide a comma-separated list for ngclient_e2e_test_solutions and have the needed solutions in the solution workspace repo.

For each tested solution (in this example there is only one - 'solution1') the subdirs 'spec' and/or 'selenim' should contain a folder with the same name as the tested solution. So for each tested solution, the ant script looks in 'spec/[solutionName]' and 'selenium/[solutionName]' subdirs and runs all the files ending in '_spec.js' (protractor tests in 'spec') or '_test' (selenese tests in 'selenium'). For the sample repository, all needed test scripts are already there.

See UI Testing with Selenium and NG Client UI Testing with Selenium for more information about creating test scripts.

(optional) 5.1 Running end-to-end tests remotely using Sauce Labs testing cloud

The above configuration will run tests using a locally installed browser (chrome, but it can be changed to others). You might find it useful to run your tests against a testing cloud that offers a variety of versions/devices/OSes/browsers like Sauce Labs.

In order to enable sauce labs testing (note that sauce configurations/options historically need to be updated from time to time in order to work so you might need to do some extra tweaking if this wiki page is not always up-to-date) you have to do some extra steps:

  • in 'Invoke Ant -> Advanced' expand the 'Properties'. Add the following lines after what is already there (replace the content between brackets):

    e2e.browser.type                    = remote
    sauce.labs.user                     = [your_sauce_labs_username]
    sauce.labs.token                    = [your_sauce_labs_token]
    sauce.labs.region                   = [one_of_eu/us;_only_needed_really_if_you_are_going_to_run_your_tests_in_Sauce_Data_Center_from_Europe_in_which_case_you_should_use_the_value_eu]
    SELENIUM_PORT                       = ${SELENIUM_PORT}
  • under 'Build environment' enable 'Sauce labs support'. Under "Scauce labs options" add your credentials (your Sauce labs account credentials; make sure - if you are going to use the sauce labs test cloud in Europe - to select EU for 'Sauce Data Center' when creating your credentials) and check "Use latest version of Sauce Connect". Under 'Sauce Connect Advanced Options' click the 'Advanced' button and check 'Enable Verbose Logging' (optional).
  • (optional) add a 'Run Sauce Labs Test Publisher' post build action.

Step 6 -- Configure/enable Deployment to Manual Test Server

If you wish the Jenkins job configured above to also deploy 'solution1' to an already running Tomcat, you can enable that by going to the job configuration page and:

  • in 'Invoke Ant -> Advanced' expand the 'Properties'. Add the following lines after what is already there (replace the content between brackets):

    build.deploy                        = yep
    solutions_to_deploy                 = solution1
    deploy.tomcat.manager.url           = http://localhost:8084/manager/text
    deploy.tomcat.username              = tomcat
    deploy.tomcat.password              = tomcat@servoy

This assumes that the Tomcat server to deploy the solution to is already running and is configured to accept connections on port 8084. It also has a user configured named "tomcat" with that has the rights to deploy a war file through the manager app (has role 'manager-script'). See how the e2e (end-to-end) tomcat is configured in software_factory_build_tools\apache-tomcat-8.0.32\conf\tomcat-users.xml and software_factory_build_tools\apache-tomcat-8.0.32\conf\server.xml.

Tomcat shut-down ports

If this tomcat is running on the same machine/host, please make sure it's shutdown port from server.xml doesn't match the one the e2e tomcat server has (8005) because then the end-to-end testing step will shut down this deployment tomcat server instead of the one meant for e2e tests.

Customizing the Jenkins Job to suite Your set of test solutions

Congratulations! Now you should have a job that runs smart and mobile client unit tests, e2e tests and then deploys a solution to the manual test server.

Next you will probably want to configure the same for your own set of solutions. Here is how you can do that.

Basic Customization

All the steps for creating jobs for your own solutions are almost identical to the sample job. Enable only the steps you need as shown above. What you should do differently:

  • Under 'Source Code Management', instead of configuring sample solution repository contents (software_factory_sample_solutions), configure the location of your own solutions to be checked out. Make sure the resulting job workspace contents are similar (after clone/checkout the job workspace directory should contain 3 subdirs: a directory that is your repository (a workspace with your solutions), the software_factory_build_scripts and if you want e2e tests then software_factory_build_tools as well).
  • Adjust 'Abort the build if it's stuck' value depending on how long you expect your test job to run.
  • At Build > Add build step > Invoke Ant > Advanced > Properties > expanded - edit in that text-area the following properties to specifiy the name of the subdir that is your workspace and the names of the root solutions relevant for your test job:
    • update the name of the workspace (containg solutions) subdirectory that you checked out above instead of software_factory_sample_solutions:

      workspace.solutions.dir   = ../../my_workspace_subdir_name
    • for (smart client) unit tests, if enabled - replace sample tests root solution names below with your own; comma separated list:

      smart_test_solutions      = my_test_solution1,my_test_solution2
    • in case of mobile client unit tests, if enabled - replace sample root solution names below with your own; comma separated lists:

      mobile_test_solutions     = my_test_mobile_solution_Tester
      mobile_service_solutions  = my_test_mobile_solution_Tester_service
      mobile_to_deploy          = my_test_mobile_solution
      mobile_to_deploy_services = my_test_mobile_solution_service
    • for end-to-end tests, if enabled - replace sample tests root solution names below with your own (comma separated list); for each of those solutions you have to write your protractor and/or selenese test scripts and put them in [my_workspace_subdir_name]/build_files/e2e_test_scripts; protractor test scripts are in spec/[solution_name] and the selenium test scripts are in selenium/[solution_name]; see Enabling End-to-End tests step above for more info

      ngclient_e2e_test_solutions          = my_e2etest_solution1,my_e2etest_solution2
    • for deployment to manual test server update the name of the solution to deploy and the manual test tomcat's properties as needed

      solutions_to_deploy                 = my_solution_to_deploy1,my_solution_to_deploy2
      deploy.tomcat.manager.url           = http://myManualTestTomcatMachine:8089/manager/text
      deploy.tomcat.username              = mydeploymentuser
      deploy.tomcat.password              = mysecretdeploymentpassword
  • Configure the servoy.properties files to be used by Servoy at each step. In your workspace repository you should have a dir/project named build_files. Create the properties files in [my_workspace_subdir_name]/build_files/servoy_properties_files/* as needed by your solutions (define servers and other needed changes; the repository server can be a Hslq (in memory) server to always import clean or you can change it to a persistent server as well). The files are:
    • exporter_servoy.properties - the export tool is in fact a kind of developer without UI. It uses this as servoy.properties file when starting up. This is used when building the servoy projects and checking for markers.
    • unit_test_servoy.properties - this is used as servoy.properties when running smart and mobile client unit tests.
    • e2e_test_servoy.properties - this is used as servoy.properties when running e2e tests (this is the properties file that the exporter will include in the exported war file); always set in this file servoy.ngclient.testingMode=true
    • deploy_on_success_servoy.properties - this is used as servoy.properties for deploying to manual test server (this is the properties file that the exporter will include in the exported war file)

Advanced Customization

The 'Basic customization' steps should be enough for most cases. If more aspects of the build require tweaking, like changing arguments used by exporter and test clients, modifying more timeouts that don't suit your solutions, running mobile unit tests on SauceLabs cloud instead of locally or changing protractor/selenese runner configurations, it can be done. For these steps you will need to clone/fork the software_factory_build_scripts repository, make your own changes to it and use it instead in the job config. Here are a few more things that you can do (of course you could do much more with what Jenkins, ant, Servoy and the other tools have to offer):

  • customize the job ant build's behavior by defining more properties in BuildAdd build stepInvoke AntAdvancedProperties > expanded. Customizable properties are listed in the first section of software_factory_build_scripts/servoyAntScripts/ant_build.properties and are explained in the comments preceding each one:

    # ------------------------------------------------------------------------------------------------
    # Properties that you should set in Jenkins job configurations (ant advanced config; they will ---
    # be available to ant and also have precedence over what is defined in this file               ---
    # ------------------------------------------------------------------------------------------------
    
    # Enable parts of the build - by default all are disabled (you can run import-test client unit tests, mobile solutions unit tests, e2e tests or just export and deploy solutions)
    #build.run.unit.tests                = yep
    #build.run.mobile.unit.tests         = yep
    #build.run.e2e.tests                 = yep
    #build.deploy                        = yep
    
    # Servoy developer installation that will be used for exporting/testing (a separate install
    # is recommended for each Jenkins job in case multiple jobs are configured to run concurrently)
    
    servoy.install.dir                  = [your_jenkins_dedicated_servoy_installation_dir]
    
    # 'workspace.solutions.dir' property should be set to point to where your solutions can be found (equivalent to the developer workspace)
    
    workspace.solutions.dir             = [the_root_of_the_workspace_containing_solutions]
    
    # 'servoy.properties.dir' is a dir that contains the servoy.properties files to be used at each step during the build:
    # exporter_servoy.properties - the properties file that the exporter will use internally when started
    # unit_test_servoy.properties - for the jsunit tests
    # e2e_test_servoy.properties - for the e2e tests 
    # deploy_on_success_servoy.properties - for the war file that should be generated for deployment
    # these can be overridden by directly setting each of the "[xyz].servoy.properties" (see below) props instead of "servoy.properties.dir" 
    
    servoy.properties.dir               = ${workspace.solutions.dir}/build_files/servoy_properties_files
    
    # Solutions to be exported and then tested;
    # multiple solutions can be specified via a comma separated list of solution names
    
    smart_test_solutions                = solution1,solution2
    
    mobile_test_solutions               = mobile_ipAddressValidatorTester
    mobile_service_solutions            = mobile_ipAddressValidatorTester_service
    
    mobile_to_deploy                    = mobile_ipAddressValidator
    mobile_to_deploy_services           = mobile_ipAddressValidator_service
    
    ngclient_e2e_test_solutions         = solution1
    solutions_to_deploy                 = solution1
    
    # By default, solution and resources projects reside directly in the <workspace.solutions.dir>.
    # Uncomment the following to send '-pl' argument to workspace solution exporters: it enables searching for
    # solution (and resources) projects inside all subfolders (only the first level) of <workspace.solutions.dir>. 
    # Example: if the workspace needs to contain projects from different git repositories, those can then be
    # checked out in '<workspace.solutions.dir>/repo1', '<workspace.solutions.dir>/repo2' and so on
    
    #alternate_project_locations.on      = yep
    
    # Uncomment the following line/set this if you want exports that passed unit testing (generated by previous
    # build) to NOT be deleted at the beginning of each build cycle; you probably won't ever need to keep those
    # unless some tool watches those folders and you want to avoid triggering something due to those deletes
    
    #dont_clean_passed_exports           = yep
    
    # This URL is used when running mobile unit tests - as the mobile test war is deployed there (the install from servoy.install.dir above)
    
    servoy.install.url                  = http://localhost:8080
    
    # Selenium server jar used by mobile test client
    
    selenium.server.jar                 = [path_to_downloaded_selenium_server_jar]
    
    # SauceLabs credentials for e2e tests (and mobile unit test if needed)
    #sauce.labs.user                     = [your Sauce Labs user here]
    #sauce.labs.token                    = [access token goes here]
    #sauce.labs.region                   = [us (for United States) or eu (for Europe)]
    
    # For testing mobile clients in SauceLabs cloud: uncomment relevant lines in
    # "run_mobile_client_tests_if_exported_ok" target of jenkins_build.xml; other changes might be needed as well, but this is the main idea
    
    # remember these properties apply to mobile client jsunit tests
    #sauce.labs.selenium.url             = http://${sauce.labs.user}:${sauce.labs.token}@localhost:${SELENIUM_PORT}/wd/hub
    # SELENIUM_PORT used above is set by the the sauce jenkins plugin
    # define a timeout for sauce labs to not require additional Selenium commands for a solution (set to 1.5 min; should be updated for how long your test can possibly take to run, but not much higher)
    #sauce.labs.selenium.idle-timeout    = 90
    
    # If you want your mobile solutions deployed (war copied) to a Servoy server after testing is done,
    # uncomment / set values as in the following two lines and change the value of
    # mobile.deploy.servoy.install.dir to point to it; passed .war files will be copied to mobile.deploy.webapps.dir location.
    
    #mobile.deploy.servoy.install.dir           = [your_mobile_war_deployment_servoy_installation]
    #mobile.deploy.webapps.dir                  = ${deploy.servoy.install.dir}/application_server/server/webapps
    
    # 1 hour time-out; edit this as needed - depending on how long it usually takes for your solutions to run unit tests (should probably be lower in most cases)
    
    unit.test.timeout                   = 3600000
    
    # 'e2e.scripts.dir' points to where you keep the e2e (selenium/protractor) scripts that should be used with your solution(s)
    
    e2e.scripts.dir                     = ${workspace.solutions.dir}/build_files/e2e_test_scripts
    
    # 'e2e.tomcat.start_stop.tools.tomcat' can be set/uncommented if e2e tests should use the tomcat from 'software_factory_build_tools' repo
    # if you use an already-started and configured (user with deploy rights) tomcat instance leave this commented out and change the other
    # e2e.tomcat.xyz properties as needed
    
    #e2e.tomcat.start_stop.tools.tomcat  = yep
    
    # the following e2e tomcat settings are by default matching the tomcat from 'software_factory_build_tools' repository
    
    e2e.tomcat.dir                      = ${antScripts.dir}/../../software_factory_build_tools/apache-tomcat-9.0.16
    e2e.tomcat.base.url                 = http://localhost:8083
    e2e.tomcat.manager.url              = ${e2e.tomcat.base.url}/manager/text
    e2e.tomcat.username                 = tomcat
    e2e.tomcat.password                 = tomcat@servoy
    e2e.tomcat.java.home                = ${JAVA_HOME}
    
    # DEPLOY tomcat settings (used if build.deploy is set) - this must be a running TOMCAT who's given user has the right to upload wars
    
    deploy.tomcat.manager.url           = http://localhost:8084/manager/text
    deploy.tomcat.username              = tomcat
    deploy.tomcat.password              = tomcat@servoy
    
    
    # This property can be remote (used for sauce labs) or local; based on this different protractor and selenese runner configurations are selected
    e2e.browser.type                    = local
    
    # ------------------------------------------------------------------------------------------------
    # Other properties that you can change but you can use defaults as well --------------------------
    # ------------------------------------------------------------------------------------------------
    
    # Paths to other downloaded tools that we need for mobile unit testing; chrome driver is only needed if not executing mobile tests in SauceLabs cloud, but locally
    
    webdriver.chrome.driver             = [path_to_the_downloaded_selenium_chrome_driver_executable]
    
    # used directories / files / settings 
    
    # If you are using a Servoy version < 7.3 please uncomment the following (otherwise leave it as it is;
    # setting it to "false" or "nope" is not the same as commenting it); the old way should work on 7.3 or
    # newer as well, but not the other way around; new way only gets rid of some unneeded exporter
    # initialization time in case multiple solutions need to be exported
    
    #servoy.install.lower.then.7.3       = yep
    
    antScripts.dir                      = ${basedir}
    results.dir                         = ${antScripts.dir}/../buildResults
    
    jsunit.results.dir                  = ${results.dir}/jsunit_results
    e2e.results.dir                     = ${results.dir}/e2e_results
    export.results.dir                  = ${results.dir}/exports
    
    export.logs.dir                     = ${export.results.dir}/logs
    jsunit.logs.dir                     = ${jsunit.results.dir}/logs
    e2e.logs.dir                        = ${e2e.results.dir}/logs
    
    export.servoy.properties            = ${servoy.properties.dir}/exporter_servoy.properties
    jsunit.servoy.properties            = ${servoy.properties.dir}/unit_test_servoy.properties
    e2e.servoy.properties               = ${servoy.properties.dir}/e2e_test_servoy.properties
    deploy.servoy.properties            = ${servoy.properties.dir}/deploy_on_success_servoy.properties
    
    smart.test.exports.dir              = ${export.results.dir}/smartExports
    mobile.export.dir                   = ${export.results.dir}/mobileExports
    e2e.export.dir                      = ${export.results.dir}/e2eExports
    deploy.export.dir                   = ${export.results.dir}/deploymentExports
    mobile.test.war.dir                 = ${mobile.export.dir}/war
    mobile.service.export.dir           = ${mobile.export.dir}/service
    
    # the following 2 are just temporary locations
    mobile.deploy.w.dir                 = ${mobile.export.dir}/prepare_deploy/war
    mobile.deploy.s.dir                 = ${mobile.export.dir}/prepare_deploy/service
    
    # this folder can be monitored by the separate "deployment" Servoy installation
    # so that when a .servoy changes in it it gets automatically uploaded (see wiki page "Building a Software Factory")
    passed.exports.dir                  = ${results.dir}/passedExports
    passed.sc.exports.dir               = ${passed.exports.dir}/smart_client
    passed.mc.exports.dir               = ${passed.exports.dir}/mobile_client
    
    lib.dir                             = ${jsunit.results.dir}/temp/lib
    
    servoy.developer.dir                = ${servoy.install.dir}/developer
    servoy.app_server.dir               = ${servoy.install.dir}/application_server
    
    ant_contrib.jar.location            = ${basedir}/ant-contrib/ant-contrib-1.0b3.jar
    
    e2e.testing.dir                     = ${antScripts.dir}/e2e
    # war deployments Servoy app. server log files will be generated in below location
    e2e.servoy_user_home.dir            = ${e2e.results.dir}/temp/servoy_user_home
  • workspace exporter arguments (you should have available all the options present in UI Solution Export Wizard) can be customized by modifying software_factory_build_scripts/servoyAntScripts/exporter.xml's 3 macrodefs named 'export-*'. You can see a list of available command line arguments at Command line utilities for Solution Exporting and Unit Testing

  • smart unit test client and mobile unit test client arguments (which are actually system properties sent to a junit test suite) can be customized by modifying software_factory_build_scripts/servoyAntScripts/unit_test_runner.xml's 2 macrodefs named 'run-*'. You can see a list of available arguments by commenting out the '<!--arg value="-help" /-->' lines in these marcrodefs and creating/executing an ant task that calls them from the command line (or alternatively just comment out those lines in a Jenkins job and check the job build's console - after it is executed). The arguments can also be inspected at Command line utilities for Solution Exporting and Unit Testing

  • end-to-end tests can be further customized by changing the protractor and/or selenese runner configurations. See software_factory_build_scripts/servoyAntScripts/e2e/servoyConfigurator.js. To change other protractor (http://www.protractortest.org/#/) settings see file software_factory_build_scripts/servoyAntScripts/e2e/protractor.config.js.template.local or software_factory_build_scripts/servoyAntScripts/e2e/protractor.config.js.template.remote depending on what environment you are running the tests on (sauce cloud or local). It is currently a template because before each run the ant script injects the sauce labs credentials specified as ant properties. To change selenese-runner settings (https://github.com/vmi/selenese-runner-java) that are used for running selenese tests, see files software_factory_build_scripts/servoyAntScripts/e2e/selenese.config.json and software_factory_build_scripts/servoyAntScripts/e2e/selenese.properties.template.localsoftware_factory_build_scripts/servoyAntScripts/e2e/selenese.properties.template.remote.

Running all this locally for development (outside of Jenkins)

This chapter shows you how you can run any of the build steps above locally. This is useful when developing, writing tests or when trying to debug a failure that happened on Jenkins.

  1. Clone the 2 or 3 repositories (the ones from initial job config + the one from e2e tests) locally to the same parent dir so that the directory configuration is the same as in the Jenkins workspace.
  2. Make any other preparations just as before that were not directly related to Jenkins. (npm, ant, Servoy Developer, JDK should be available)
  3. Make a local copy of the file software_factory_build_scripts/servoyAntScripts/local_ant_build.properties.template. Name it local_ant_build.properties. In that file you should have whatever you used to set in Build > Add build step > Invoke Ant > Advanced > Properties > expanded, so just replace the whole file content with that. (using copied properties from the template is only useful if you want to start tweaking from scratch)

  4. Run the ant script (target 'main'). Assuming ant is installed and available in path, go to directory software_factory_build_scripts/servoyAntScripts and run 'ant -f jenkins_build.xml'.
  • No labels