Running docker on WSL (Windows Subsystem for Linux)

UPDATE 2019/11/22:
There is a much better way to do this now with WSL2.  You can actually host the Docker engine inside WSL2 and have it accessible to windows & Ubuntu without needing to expose without TLS. If you did the steps below previously, remove the export DOCKER_HOST declaration before updating to the new WSL2 version.
https://docs.docker.com/docker-for-windows/wsl-tech-preview/

————

My build toolchain is frequently in there for node based applications, and I was looking for a way to run docker commands from WSL (Windows Subsystem for Linux https://docs.microsoft.com/en-au/windows/wsl/install-win10 ).

Note: The below steps are a quick way to get it working but involves disabling TLS authentication. A more involved way to do this without disabling TLS can be found at https://blogs.technet.microsoft.com/virtualization/2017/12/08/wsl-interoperability-with-docker/

This setup puts the docker engine in Windows on top of Hyper-V, but allows you to control it either from Windows, or within WSL by connecting it to the Windows Docker Engine.

1. Install docker for windows https://docs.docker.com/docker-for-windows/install/
2. Go into the Docker for Windows settings and expose daemon on localhost
   
3. Install docker in WSL (it’ll just install the client) https://docs.docker.com/install/linux/docker-ce/ubuntu/
4. Tell WSL to look for a docker engine on localhost
   TIP: put the export into your ~/.bashrc so you don’t need to run it ever time.
   export DOCKER_HOST=tcp://127.0.0.1:2375

Now you will be able to run standard docker commands run new images, show the running containers, etc.

Ethereum DevOps with VSTS – easier now with new Truffle installer + npx

Last year I blogged about my first attempt to wrap a DevOps flow around Ethereum development. Since then there have been improvements in the way that Truffle is deployed with npm, the npx command that comes with npm 5.3.0 onwards, VSTS now supporting hosted Linux build agents, but also my familiarity with Linux, npm & Truffle have increased. I have spent a few weeks in my spare time trying to vastly streamline and simplify getting a basic pipeline going that provides: compilation, testing, migrating of solidity contracts using Truffle.

The steps I outline below will work on any build system, such as Jenkins or Team city that can run bash commands. But I have shown VSTS as that is what I am most familiar with.

Screenshot of the final result, with VTSTS showing Truffle test results for each build.

A video where I walk through the entire process end to end.

Example configured project

I have created a repository in GitHub where I have done the below steps. So if you want to just clone it and try it out on your own build server, that will make it easier for you

https://github.com/DavidBurela/truffledevops

1. Configure npm development packages for your Truffle project

Add Truffle, TestRPC, Mocha & Mocha JUnit plugin as DevDependencies in your packages.json. This will allow the build server to later install these packages and then execute Truffle commands.

# don't run npm init if your project already has a packages.json
npm init -y
npm install truffle mocha mocha-junit-reporter --save-dev

2. Configure truffle.js

a) remove development network. Truffle test will spin up its own temporary network if there isn’t a development network specified. Just make sure to remove the development network definition if one exists.

b) Mocha test reporter definition. We leave it as “spec” so that it will display the results in the console window while developing locally. Later on the build server we can change the reporter to JUnit, and the output file is already specified here ready for that.

module.exports = {
//...
// add a section for mocha defaults
mocha: {
reporter: "spec",
reporterOptions: {
mochaFile: 'TEST-truffle.xml'
}
}
};

 

3. Build server settings

This screenshot shows what the end result looks like. It can be summarised as: get the tests, install the npm packages, configure any environment variables, run the Truffle commands, and collect the test results.

a) Get sources

Define where your code is sitting (e.g. VSTS, Github, Bitbucket, etc.)

b) npm install

The default npm install task. Will look in packages.json and install our dev dependencies so later we can run Truffle, TestRPC, etc.

c) Shell script – environment details and config

An inline script that spits out useful environment information to help debug if things go wrong.

It also importantly replaces the line in our truffle.js to change the test reporter to the console UI, to output the results into a .xml file in JUnit format

# output version details for debugging
node -v
npm -v
npx truffle version

# string replace the mocha reporter to junit output
sed -i -e 's/reporter: "spec"/reporter: "mocha-junit-reporter"/g' truffle.js

d) Shell script – truffle commands

Inline script that executes the local version of Truffle with npx. I like have all 3 lines, so if something breaks we can see where it happened.

# check the contracts compile
npx truffle compile

# run unit tests
npx truffle test 

e) Publish test results

Default task. I have left the defaults

How to install Jekyll on Windows 10 with “Windows subsystem for Linux”

I previously wrote how to install Jekyll on Windows by installing the Windows version of Ruby and then installing the gems that way. I have found another way install Jekyll via the Ubuntu version of Ruby. This is my preferred way now, as the Linux version of these tools are updated more frequently than the Windows versions.

 

1. Install Ubuntu bash on Windows

1. Enable Windows subsystem for Linux.
   Follow this short guide on how to enable it https://msdn.microsoft.com/en-us/commandline/wsl/install_guide
   
2. After following the steps in the guide above. Simply start the Ubuntu bash shell
   

 

2. Install Ruby & Jekyll

# Get Ubuntu up to date and install Ruby
sudo apt-get update -y && sudo apt-get upgrade -y
sudo apt-get install -y build-essential ruby-full

# update ruby gems and install Jekyll
sudo gem update –system
sudo gem install jekyll bundler

Then you can do the standard ‘jekyll new foldername` and `jekyll serve` to host it.

FYI: the error message in the screenshot about Bash on windows, no longer applies. As the Windows 10 Creators Edition resolved this issue

Easily install Jekyll on Windows with 3 command prompt entries and Chocolatey

UPDATE 2017/05/17:
I now recommend using the “Windows 10 Subsystem for Linux” which provides an Ubuntu bash shell, to install and use Jekyll. This is how I use it myself.
Install guide is here https://davidburela.wordpress.com/2017/05/17/how-to-install-jekyll-on-windows-10-with-windows-subsystem-for-linux/

 

I’ve been using Jekyll to create static websites on GitHub Pages, but I was unsure how to install it with dependencies I wasn’t used to (NodeJS & Ruby). Here is the easiest way to get Jekyll set up on your machine in just a couple of minutes.

TL;DR if you have Chocolatey installed the 2 commands are:

1. choco install ruby -y
2. gem install bundler
3. gem install jekyll

 

Prerequisite – Chocolatey 

You need to have Chocolatey installed on your machine. Chocolatey is the BEST way to install and keep applications updated on windows.

1. Open a command prompt with Administrator access
2. Install Chocolatey

   @powershell -NoProfile -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"

3. Close the command prompt as Chocolatey will not be available until you close and reopen.

Install dependencies & Jekyll

1. Open a command prompt with Administrator access
2. Install Ruby
   choco install ruby -y
   
3. Close and open a new command prompt with Administrator access
4. Install gem bundler
   gem install bundler
5. Install Jekyll
   gem install jekyll
   
6. UPDATE 2016/12/5: if you get an error about SSL, you will need to manually update Ruby Gems, because their certificate expired… Hopefully the new certificate will be bundled in the future
   http://guides.rubygems.org/ssl-certificate-update/
   

Now you can use standard Jekyll commands to create a new site and serve it e.g.
jekyll new myblog
cd myblog
jekyll serve

 

Edits:
2016/12/05: Added details on how to get resolve the Ruby Gems certificate expiring. And added gem bundler which is a new requirement.
2015/12/05: From Jekyll 3.0 you do not need to install NodeJS. This brings it down to just 2 command prompt entries to install Jekyll