How to build in Jenkins using Netbeans and Mercurial DCVS on bitbucket

by


Posted on


How to build in Jenkins using Netbeans and Mercurial DCVS on bitbucket image

As posted earlier, we use the combination as mentioned in the title to create an automated build system so we can do nightly,RC, and final build releases. But, To come this far I had some troubles to get Jenkins working with our project. Some stuff was just plain obvious, bot some others where not. In this post i can maybe help some to get your Netbeans -> Mercurial -> Jenkins combination working. To go further with this post, i assume you already have Mercurial (tortoisehg installation) working within Netbeans, and you are happy committing, pushing and pulling your files. I base this post on Netbeans 7.3.1, and tortoiseHG 2.9 (Including Mercurial 2.7). Today I updated Jenkins to version 1.529, so this is the version i'm showing in this post. Also the below instruction are partially compatible with the Netbeans Hudson plugin.

Setup Jenkins to work with your netbeans installed JVM and ANT

Resolving JUnit not found message in jenkins is at the bottom of the page.

We do some builds and tests in netbeans. This means we use the in netbeans configured JVM and ANT. To have the same configuration you should point jenkins to the same information. Because This jenkins install is a test install on my local computer, all the paths are pointing on my local machine. If your jenkins install is on an other pc/server/whatever. it would be handy to install of course the same JVM over there and include a netbeans install. Change the paths accordinally the below setup.

First, point to the same JVM you are using in netbeans. In netbeans right click your project. Click Libraries and Manage as shown below:
nbchkjvm

  • In Jenkins go to the dashboard,
  • click Manage Jenkins and then Configure System.
  • Scroll down until you see JDK. Click the button to add JVM’s As shown in the screenshot below, i added the JVM i have configured in Netbeans. Unselect the “install automatically “.
  • For the ANT configuration, scroll down until you come across Ant, click the button to add, give the installation a name, and put the path to the netbeans Ant installation folder. The screenshot below shows my configuration. You probably have to change your paths to match your installations.

jenkinsjvmplusantpaths

You’re done matching your netbeans configuration in jenkins.

Set project configuration to match netbeans

To use the above configuration,

  • go to your project configuration via Dashboard,
  • click your project, and click configure.
  • Scroll down until you come to the JDK version to use, select the appropriate version

jenkinsmatchjdk

Scroll down until you come to Build, add a build step with the “Invoke Ant task” option. Select the appropriate Ant configuration as shown below
jenkinsprojectselectanttask

As you can see, i have set a build file. Because in our server repository we have multiple projects, with one of them being the pidome-server, i had to specify the correct build.xml. At total the server project has 4 projects in the repository, and each project has an ant build task. This post does not cover this.

Set up Jenkins with Mercurial

Our repository is hosted on bitbucket, but as of this moment, still private. To get get jenkins to do Mercurial, you first have to install the mercurial plugin.

  • On your jenkins dashboard click “Manage Jenkins” and then “Plugin Centre”.
  • There is a tab called “Available” scroll down until you get to the Mercurial plugin. Select it.
  • Click “Install”. On the screenshot below you can see that i already installed the plugin.

jenkins_instal_merc

If you have done the installation, you can continue to configure your bitbucket mercurial repository. Mercurial is already on the system path if you have already installed tortioseHG with mercurial on windows and rebooted. If the installation of mercurial is on Linux, you first have to set the mercurial installation path at the system configuration.

  • Go to dashboard -> manage jenkins -> configure system -> scroll to Mercurial.
  • Add a mercurial installation name, path and executable.

As said earlier, our repository is private at the moment. So we need authentication before we can pull. There are two ways you can accomplish this. The easy insecure way and the more difficult more secure way. I will explain both.

Using authenticated bitbucket repository pull the easy way

On your Jenkins dashboard, click your project and go configure. Scroll down until you see Source Code Management. Here you supply your mercurial repository with you username and password as shown below in the screenshot.
jenkinsmerceasyauth

Test current setup

The above configuration worked for me at my part. Chances are big this will also work for you. If you just click “Build” when you saved the configuration, clicking on console will give the output. This setup has a disadvantage because the mercurial plugin sees different url’s. This means that instead of doing an update of the repository, it always pulls the complete repository from scratch. I have not tested a build trigger with SCM polling, because i think it will always build because of this.

Using authenticated bitbucket repository pull with the less easier but some more secure way with ssh

If you want to publicize your build results, i don’t think you are comfortable with having your username and password shown in the build log. So, we are going to setup jenkins to use a more secure method to pull the repository using ssh.

To communicate over ssh without entering a username and password we need to create ssh keys.  This little howto is different then shown on https://confluence.atlassian.com/display/BITBUCKET/Use+the+SSH+protocol+with+Bitbucket. The one at bitbucket shows how to use ssh with a private key which uses a pass phrase, a protection for your private key. As Jenkins is a deployment server which does not communicate with an end user when it is doing a pull from the repository, a pass phrase can not be entered, so the server “hangs”. If it did communicate with the end user then there should always someone be present at the server to enter the pass phrase when the server connects with the repository. We are going to create a private key without a passphrase. Because this is a security thread we are going to use the bitbuckets feature deployment keys, and not using the key as a personal key.

Create key

Before continuing create a .ssh directory in your user directory, or an other directory you feel reasonable safe to place your unprotected private key. To make a directory like this windows won’t always allow you to create a directory starting with a dot. To do this.

  • Open a “dos” box with: “windows key + r”,
  • type “cmd” press enter.
  • type “cd c:\users\yourusername” where yourusername is your windows username.
  • Type “mkdir .ssh”.
  • You now have a directory with a dot. type exit and then enter to close the “dos” box.

If you do not already have, download puttygen from the putty download page. Start the program and click generate, you will have to use your mouse over a empty field to generate the key as shown below:

untitled-1

 

  • Click on save private key. You will get a message if you are sure you save a private key without a passphrase. Klik Yes.
  • Browse to the location where you earlier created the .ssh directory. Give it a name which ends in .pkk . Do not yet close the key generator.
  • Open a browser, and go to your bitbucket repository.
  • Go to the repository settings page. On the left menu you have a link with Deployment Keys.
  • Click “Add key” As shown in the screenshot below, give it a name. Copy the public key which in the above screenshot is marked to the Key field.
  • Click “Add key”.

bitbucketsetpublickey

You can now close the key generator and bitbucket.
You can use the same public deployment key with more repositories. We use the same (not the one shown in the example off course) for multiple projects (server and client).

We are now going to configure TortoiseHG to use the private key to connect with bitbucket.

  • Start TortoiseHG Workbench and go to file and then Settings.
  • In the popup click “Edit file” on the right.
  • This opens the mercurial.ini from your user directory.
  • Put the next in this file at the ui section:
[ui]
ssh="C:\program files\tortoisehg\TortoisePlink.exe" -ssh -2 -i "C:\Users\computerusername\.ssh\privatekeyname.rsa.ppk"

Replace the path with your private key location and name. Be aware that the above solution causes any connection to use this private key. It still not works, as with every first ssh connection to a new server, you will receive a host key. This needs user interaction. So before to continue you need to accept the key first. Open a dos box (“windows key + r”, type cmd, press enter). Type

"C:\program files\tortoisehg\TortoisePlink.exe" hg@bitbucket.org

Accept the host key. You are now done with prerequisites to start using jenkins with bitbucket. The only thing that remains is to let jenkins mercurial plugin know that it has to use the TortioseHG installation so it can start using the ssh connection.

Modify jenkins and project configuration

Go to your jenkins installation and go to Manage jenkins and Configure system. Scroll down to Mercurial and add a mercurial installation like in the screenshot below, you should change the installation directory according to yours and save the settings.

jenkins_merctortoisehg

Go to your jenkins job, and to the configuration screen. Scroll down until you are at Source Code Management. Change your setup as shown below in the screenshot
jenkins_finalmercsetup

Notice that the repository url now starts with ssh instead of http. This is the exact url as you can find on bitbucket when you are on your repository summary page and click the “Clone” button, and change the setting to ssh. You should only copy the ssh url part. Save the new settings.

If you are running jenkins as a service in windows its highly possible it still does not work, try to build, if it doesn’t, come back, there is more.

If it still does not work

If you are reading this, it didn’t work, or you are continuing reading. The possible reason it didn’t work, is because i think the system account is not allowed to run the hg.exe from the tortoisehg installation. The next steps will make some user rights changes as needed on my system to make it work.

  • Go to your windows control panel, open the Administrative tools, and open Services.
  • scroll down the list until you see jenkins. double click it. You will now get to the properties of this service. I suggest you change the user under which this service runs under in your username.
  • Click on the Tab “Log On” You will see that the system account is selected. Select “This Account” and click “Browse”.
  • In the popup type the username as shown below in the screenshot (in my case John).

changeservicelogon

  • Press enter, your username should be set in the field, press ok.
  • Type your password twice in the properties screen. Press ok again.
  • You will now probably get the message that your account now can run services.

You are almost done. If you have installed Jenkins in c:\program files\ this is the last step.

  • Open explorer and browse to c:\program files\.
  • Right click the jenkins folder and go to properties.
  • Click the tab “Security”, press “Edit”. In the permissions popup click “Add”.
  • You will now see the same popup as you have seen in the service properties.
  • Type the same username again, as seen in the screenshot below.

jenkinsfolderusermod

  • Press Ok,
  • again Ok and
  • Ok again.

You will now see windows adding your user to the files.

Now the keys are set, Jenkins and the Mercurial plugin is configured, and the user rights are set on the service and the files. For me this was the solution to get it working. It seems like a lot, bit it really is done in a couple of minutes. Now you should also be able to run the builds, after rebooting. Restarting the service did not help me.

John.

P.S. keep on reading if you have a problem running unit tests in Jenkins with the message “org.junit does not exist”.

Resolve Unit test problems in jenkins with netbeans projects with JUnit

When unit test fail with “org.junit does not exist” while it should, the problem is fixed quite easy. Some background: The problems lies in the fact that netbeans does some injection in the nbproject/project.properties file with netbeans configuration items. One of these configuration parameters is: “${libs.junit_4.classpath}”. If you do not want to fiddle around with the build.xml or other Ant xml files. You just have to change one little thing in your jenkins project build configuration.

Go to your jenkins project configuration, scroll down to your Ant build which has the “test” target. Click advanced and add the java option: -Dlibs.junit_4.classpath=”c:\Program Files\NetBeans 7.3.1\platform\modules\ext\junit-4.10.jar” as seen in the screenshot below
junittestchange

You have to change the path to your junit path as appropriate.

You can now start testing.

This was quite a long posting, i hope it has useful parts for you if you are working in the same combination as me and the title suggests.

Latest news/blog

Friends of PiDome

RFXCOM
Affordable 433Mhz RF transceivers
UniPi.technology
Home-automation hardware manufacturer
Atlassian
Products for teams, from startup to enterprise

Latest added technology

  • SMS
  • Z-Wave
  • PushBullet
  • PlugWise
  • MQTT

Some project stats

Open Hub project report for PiDome Platform

Other

Official NLJUG Member