Rather than sticking messing around with multiple Azure provisioned names we decided to purchase a domain name for our internal development systems. Again this was really easy to do through the Azure App Service blade and within a couple of minutes we had the name and I'd setup the A record for our SonarQube server. A quick test proved that I could now access the system over HTTPS using our new domain name. Just one problem, it was using a self-signed certificate!
After looking around I decided to try out LetsEncrypt.org to get a certificate. The biggest problem with this is that the tools they provide to get a certificate don't work on Windows. Fortunately there's a pretty good Windows utility written by a community member which works brilliantly on Windows called letsencrypt-win-simple.
To run this tool I had to temporarily disable the reverse proxy rules I'd created in IIS, make sure that the site was backed by a folder on the local drive and open up HTTP access (by binding the appropriate port in IIS and enabling access to the server of HTTP in the Azure resource groups Network Security Group), once I had this I could run the tool from the command line. Other than a couple of prompts for me such as selecting the correct website it had auto-discovered the process was completely automated and in about 15 seconds I had a valid SSL certificate installed and configured against the correct binding, the self-signed certificate had been replaced with the new valid certificate. After this it was a trivial task to then reset the changes I'd made previously such as re-enabling the reverse proxy rules, removing the HTTP binding and removing the rule in the Network Security Group.
So, SonarQube was up and running with all of my old data migrated, user accounts were set up and working and I could log in over an HTTPS connection. A quick change of settings in Visual Studio Team Services and the end-point was now pointing at the new server as well.
I ran a quick test build which ran analysis using the SonarQube quality profiles and it failed!
The error in the build output was pretty long but buried towards the bottom of it was this little gem of an error message.
So the Java process doesn't like the certificate? It came as a bit of a surprise as I'd tested the site out in a number of browsers and none of them had reported a certificate problem. A quick Google (other search engines are available) and I came across this thread on the LetsEncrypt community boards. It turns out that LetsEncrypt along with a number of others such as StartSSL are not included in the out-of-box Java client trust store. In the thread someone has posted a quick URL reader class you can use to prove the point by pointing a request at https://helloworld.letsencrypt.org and naturally this fails with the same error.
sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
There's quite a few suggestions of how to work around this on the thread but the most common theme is to add the chain certificate to the trusted store manually. I tried out a few of the suggestions in the thread but had a couple of problems.
- Some of the solutions are Linux based
- Almost all of them talk about a chain.pem file and I had no idea what that was!
All of the solutions largely focused in around the Java keytool.exe utility you can use to add a certificate to the clients trusted store, so just need to figure out which certificate to install.
I'll not cover the process I took to work it out, but eventually I got the the LetsEncrypt X3 cross signed certificate which you can get from their website.
To use the keytool.exe you need the latest Java runtime installed on the server running the Build Agent, the utility should then be available under %JAVA_HOME%\bin\keytool.exe. To try and automate the process a bit I created a PowerShell scrpit which will download the certificate and execute the keytool.exe utility to install the certificate to the trusted key store. It needs to be run as an administrator and naturally the machine needs internet access. Feel free to use or modify as needed.
Once I'd run this on the build server I re-ran the build and this time, SUCCESS! Now I just need to remember to do this each time the JRE gets updated.