Writing Truffle tests with async/await

The documentation and sample projects with Truffle use promise chaining to write the test code.  As someone from the C# world, coming into the JS world it was getting really confusing to keep track (as well as all the variables I’d have to declare outside of the promise chain to use later).

I did a bit of research and found a way to do async/await in Truffle. I have put up a small simple repo by taking the truffle init project sample, and then doing a straight 1:1 conversion of their promise chaining tests over to the async/await format to help illustrate the difference. I really like the async/await version, as it is cleaner to read and much more compact (as you can see in the before/after screenshots below).

You can find the repo at https://github.com/DavidBurela/TruffleAsyncTests

 

before – promise chaining https://github.com/DavidBurela/TruffleAsyncTests/blob/master/test/metacoin.js

image

 

after – with async/await https://github.com/DavidBurela/TruffleAsyncTests/blob/master/test/asyncmetacoin.js

image

 

A shout out to Jim McDonald for pointing me to his GitHub repo where he had implemented async tests to get me started.

Advertisements

Ethereum DevOps with VSTS – easier now with new Truffle installer + npx

Last year I blogged about my first attempt to wrap a DevOps flow around Ethereum development. Since then there have been improvements in the way that Truffle is deployed with npm, the npx command that comes with npm 5.3.0 onwards, VSTS now supporting hosted Linux build agents, but also my familiarity with Linux, npm & Truffle have increased. I have spent a few weeks in my spare time trying to vastly streamline and simplify getting a basic pipeline going that provides: compilation, testing, migrating of solidity contracts using Truffle.

The steps I outline below will work on any build system, such as Jenkins or Team city that can run bash commands. But I have shown VSTS as that is what I am most familiar with.

clip_image001
Screenshot of the final result, with VTSTS showing Truffle test results for each build.

A video where I walk through the entire process end to end.

Example configured project

I have created a repository in GitHub where I have done the below steps. So if you want to just clone it and try it out on your own build server, that will make it easier for you

https://github.com/DavidBurela/truffledevops

1. Configure npm development packages for your Truffle project

Add Truffle, TestRPC, Mocha & Mocha JUnit plugin as DevDependencies in your packages.json. This will allow the build server to later install these packages and then execute Truffle commands.


# don't run npm init if your project already has a packages.json
npm init -y
npm install truffle ethereumjs-testrpc mocha mocha-junit-reporter --save-dev

2. Configure truffle.js

There are 2 main things that are being added here:

a) TestRPC provider as a new network, with the require at the top of the file to import it. This will allow Truffle to just spin up TestRPC just for the context of the command, and then get thrown away once the command finishes. Great for testing migrations, or running tests, and then throwing it all away.

b) Mocha test reporter definition. We leave it as “spec” so that it will display the results in the console window while developing locally. Later on the build server we can change the reporter to JUnit, and the output file is already specified here ready for that.


var TestRPC = require("ethereumjs-testrpc");

module.exports = {
networks: {
development: {
host: "localhost",
port: 8545,
network_id: "*" // Match any network id
},

// add a new network definition that will self host TestRPC
localtest: {
provider: TestRPC.provider(),
network_id:"*"
}
},

// add a section for mocha defaults
mocha: {
reporter: "spec",
reporterOptions: {
mochaFile: 'TEST-truffle.xml'
}
}
};

image

3. Build server settings

This screenshot shows what the end result looks like. It can be summarised as: get the tests, install the npm packages, configure any environment variables, run the Truffle commands, and collect the test results.

clip_image003

a) Get sources

Define where your code is sitting (e.g. VSTS, Github, Bitbucket, etc.)
clip_image004

b) npm install

The default npm install task. Will look in packages.json and install our dev dependencies so later we can run Truffle, TestRPC, etc.
clip_image005

c) Shell script – environment details and config

An inline script that spits out useful environment information to help debug if things go wrong.

It also importantly replaces the line in our truffle.js to change the test reporter to the console UI, to output the results into a .xml file in JUnit format


# output version details for debugging
node -v
npm -v
npx truffle version

# string replace the mocha reporter to junit output
sed -i -e 's/reporter: "spec"/reporter: "mocha-junit-reporter"/g' truffle.js

clip_image006

d) Shell script – truffle commands

Inline script that executes the local version of Truffle with npx. I like have all 3 lines, so if something breaks we can see where it happened.


# check the contracts compile
npx truffle compile

# check migrations execute without issue
npx truffle migrate --network localtest

# run unit tests
npx truffle test --network localtest

clip_image007

e) Publish test results

Default task. I have left the defaults
clip_image008

Report: Microsoft Australia DX hackfest (July)

An important part of being a Technical Evangelist at Microsoft is continuously upskilling and playing with different technologies. Taking 2 days out a month to sit down together and hack, gives us a chance to learn from each other. For example Simon briefly mentioned that he was playing with Xamarin Forms & Android development, but was having issues with the Intel Android emulators, so I was able to quickly show him the new Visual Studio ones that run on Hyper-V. Conversely I was having issues with NodeJS that Simon & Elaine were able to help me out with.

And of course, we took the time out for our usual #TacoTuesday DE--sUTUAAAmwLR.jpg

Like our previous hacks, the Melbourne team were hosted by Frank Arrigo out at the Telstra Innovation Labs https://davidburela.wordpress.com/2017/05/25/report-microsoft-australia-dx-hackfest/. While we also had Azadeh joining in remotely from Sydney, and Hannes remotely from New Zealand. 20170718_152147(0)

David (Me) – Meme classifier

I decided to make a system that could automatically classify Internet Memes. There are whole subcultures on Reddit dedicated to them, one of my favourites being https://www.reddit.com/r/AdviceAnimals/. I wanted to use the new Custom Vision service https://CustomVision.ai/ to train it on the different meme types, and be able to upload a meme and be told which category it is.

Training the custom AI was easy, I uploaded samples that I got off Reddit and clicked train. Testing it with other images correctly identifies them. Creating and training only took 10 minutes, I spent way longer browsing Reddit looking at memes ^_^;;

custom vision trainingcustom vision test

Next I wanted to build a chat bot and allow people to upload an image, and have the AI return back the category, and send a link to the correct page on Know Your Meme e.g. Success Kid. I decided it would be a great time to try out the Microsoft Bot Framework for NodeJS. I have used NodeJS & npm to download and use Blockchain toolchains, but never developed directly on it.
I have enough time to fully build out the chat bot, but I learned HEAPS about using VS Code and debugging NodeJS apps using VS Code. Lots of little gotchas when developing with NodeJS for the first time.

 

Azadeh (remote from Sydney)

I wanted to solve the first world problem that most of us have! have I turned off my hair iron strengthener?
It turned out there are lots of people have the same problem, please read http://www.ismyhomesafe.ca/did-you-forget-to-turn-off-your-hair-straightener/ and https://www.honeywell.com/newsroom/news/2014/12/new-research-uncovers-fear-of-leaving-on-appliances-is-a-major-worry
to solve the problem I used wemo switch. I created two recipes/applets in ifttt for turning on and turning off the wemo switch. Basically, I got two endpoints for turning on and off the switch.
To make it more user-friendly and accessible, I used azure bot service and created a chat bot that can get commands to turn on and off the switch.
I used LUIS to understand intents and call the proper endpoint based on the command.
I hosted the source code on github and set continues integration to make sure after every push to master, the new code got deployed to azure bot service and updates the bot.
source code: https://github.com/Azadehkhojandi/WemoBot

 

Rian

I used Azure Cognitive Services Text Analytics to analyse Star Wars subtitles tracks. Topic Detection and Sentiment Analysis both seemed like good candidates.
Key Learnings:
1) Topic Detection doesn’t work well with many ‘documents’ of very small size (e.g. lines of subtitles), of as little as one word. A better approach was to approximate scenes and aggregate lines into larger documents.
2) Sentiment data is very noisy. A naive prediction is that such a sentiment analysis would track the cadence of the film. This is not at all the case, as you can see in the graph of the sentiment of the Phantom Menace.
3) Slang/ colloquialisms break topic detection, e.g. Jar Jar Binks’ lines like ‘mesa in trouble’. These should be excluded from the Topic Detection algorithm using Stop Words or Stop Phrases field in the request.

The plot below tracks sentiment across all pseudo-scenes throughout the film. You can see the data is highly variable and does not seem to follow the cadence of the film. A further research question might be to vary the size of pseudo-scenes (i.e. to aggregate lines into variable sized batches), and run sentiment analysis on all these pseudo-scenes. The result may better approximate the cadence of the film.

MicrosoftTeams-image (2).png

 

Hannes (remote from NZ)

hannes hololens.png

The app is made using Unity, and the HoloToolkit.
You can see how far along progress currently is in this video.
The idea is to bounce a table tennis ball on a paddle that you drag around with your hand. It has a scoreboard that tracks your high score for the session.
When you open the game, you are presented with a paddle and a ball hanging in the air above it. To start the game, you simply tap and hold on the paddle, which starts the ball falling. Keep the paddle under the ball to make it bounce. You get a point for every time the ball bounces on the paddle. Releasing the paddle resets the position of the ball.

 

 

 

New releases of Truffle & TestRPC. Now easier to install.

I have mentioned previously that Truffle & TestRPC are my development tools of choice for Ethereum based development.

truffle logo

It is exciting to see that there have been new releases of both Truffle & TestRPC. I have been hanging out in the Truffle Gitter channel, and been fielding lots of questions about installation issues on Windows. As Tim has previously blogged, there have been a number of issues with installation. The installation issues were mostly due to compiling C based libraries, and requiring a number of build tools on the local machine. These issues have now been removed as Truffle & TestRPC are both now built, and then released as a completely pre-packaged release. This also means that installation times have dropped from 10 minutes, to 10 seconds! (Which means I’ll need to go back and update some of my old tutorials )

 

Truffle

Highlights: More reliable installation. Makes it easier for everyone to jump in and start playing!

https://github.com/trufflesuite/truffle/releases/tag/v3.3.0

TestRPC

Highlights: Installs quicker. Reduced memory consumption. Ability to persist the blockchain to disk, meaning you can suspend and then resume if you like  (I’ll still be using it mostly as a throw away in memory style dev environment).

https://github.com/ethereumjs/testrpc/releases/tag/v4.0.0

Whats next?

I’ve been waiting in suspense for these updated installers to drop for many months now. There have been a couple of projects on my backlog that had been blocked until this came out. A short list of what I’ll be working on in the near term are:

Report: Microsoft Australia DX hackfest

An important part of being a Technical Evangelist at Microsoft is continuously upskilling and playing with different technologies. Each of us are usually off speaking to different customers or attending developer events, so to give us a chance to work together as a team and learn from each other we decided to set up a regular internal hackfest.

Last month we had our first, and the Melbourne team were hosted by Frank Arrigo at the Tesltra Innovation Labs. It is an awesome space, and we plan on hosting a LOT of future hackfests there. We also had our remote team mates working away and keeping in touch during the event.

20170419_115014 (2)
20170419_141929

Each of us hacked away on our own experiments, which gave us a chance to check out the latest toolchains and APIs. But it was great being able to just ask each other for advice.
At the end of the 2 days we all jumped onto a conference call and showed off what we were able to throw together. The valuable thing was just hearing the learnings from each person on the “gotchas” they discovered when working with the tools/tech.
20170419_161517

Here is a little summary of what each of us worked on and learned:

David (me)

I wanted to build a little utility that utilised the Microsoft Graph https://developer.microsoft.com/en-us/graph/. The idea being that you want to compare what distribution lists you and your peers are on, as it may make suggestions on ones that you should join (like Azure insiders). I worked through the graph documentation and used the graph explorer https://developer.microsoft.com/en-us/graph/graph-explorer/ to figure out the set of queries I would need to pull out the data I needed:

https://graph.microsoft.com/v1.0/me/memberOf  – lists distribution lists that I am on
https://graph.microsoft.com/v1.0/me/manager – gets my manager
https://graph.microsoft.com/v1.0/users/<manager email from above>/directReports – returns who my peers are
https://graph.microsoft.com/v1.0/users/<peer email>/memberOf – loop through the returned list of peers, and get each of their DL subscriptions

I’d then be able to compare the DLs that I’m on, with the ones that my peers are on. And flag which ones we have in common, and which ones we don’t share as suggestions.
The next step was to build a web app to do this. I jumped onto the Microsoft Graph quickstart https://developer.microsoft.com/en-us/graph/quick-start to generate a skeleton app as my starting point. This required registering my app on https://apps.dev.microsoft.com/ which would allow my to request permissions from the user, to access the graph on their behalf.

image

I was able to get my application to authenticate, and query details about myself and my manager. However to retrieve what DLs other people are on requires the Directory.Read.All delegated permission, and because that can potentially leak sensitive information about your organisation, only Admins can great that permission. This meant I was stuck as I don’t think the Microsoft Admins will grant me permission for my dinky little utility to run on the corporate tenant 😉
But it was still a good exercise as I was able to see each of the pieces working, and got some basic queries working.

Azadeh

Wanted to learn more about how to use Unity (as a lot of our customers are using it now for things like Hololens). She built a 2D Tetris Game, by following the tutorial at https://noobtuts.com/unity/2d-tetris-game

Tetris

Elaine

Wanted to play around with Conversations as a Platform and learn more about what is possible with bots.

On day 1 lost a fair bit of time with some visual studio 2017 issues. These seemed to be related to having a pre-release installed side by side with VS 2015 and then installing the full release version.
I intended to test more .Net core items but with the time lost I pivoted on to an area I was comfortable I could rapidly progress.

Further tested this github project which I have contributed to for generating a bot and tab for Microsoft Teams https://github.com/wictorwilen/generator-teams
I
used this to generate a Tab and Bot and deploy it to one of my development O365 tenancies. 
This included hosting the Bot and Tab an Azure tenancy and deploying via a local Git repository (via this https://docs.microsoft.com/en-us/azure/app-service-web/app-service-deploy-local-git  ). This was a nice simple option that I hadn’t used before as had previously only used VSTS and full GitHub. This was exactly as easy as expected to get running so was a nice option to tick off the list.

As part of the testing of the Tab I confirmed that I could get the Tab Theme switching working (as per https://blogs.msdn.microsoft.com/richard_dizeregas_blog/2017/02/07/microsoft-teams-and-custom-tab-theme/ ).
This worked quite well although on a slow internet connection the event firing was delayed hence there would sometimes be a several seconds of the tab showing before it changed colours to match the teams client.

Also included Office UI Fabric (https://dev.office.com/fabric ) to check if that would have any issues working in a tab inside teams. I only had time to test a few elements including the spinner but these all worked well on the Tab. The main consideration is the theming may not  automatically flow through to these elements as the Teams Style sheets have very specific classes that they target hence things need to be wrapped in the elements for them to be able to change colour when needed.  This is especially important if you want your tab to work on the high contrast setting.

Finally I tried to extend the bot via  some deep linking scenarios following this https://msdn.microsoft.com/en-us/microsoft-teams/deeplinks , however was not as successful.   Asking the bot to send the url taken via manually grabbing a deep link for the tab worked well but that had a different format to the article.

default themedark themecustom theme
Screenshots of playing with the themes

Simon

Simon being Mr. DevOps, wanted to explore combining VSTS with chat bots. Whenever a build was kicked off in VSTS, he wanted to report back if the build was successful or not, and allow users to instruct the bot to trigger Release Management to push the successful build to different environments.

He was able to get the chatbot reporting new events in a Microsoft Teams channel, and having the bot trigger certain things back on VSTS.

How to install Jekyll on Windows 10 with “Windows subsystem for Linux”

I previously wrote how to install Jekyll on Windows by installing the Windows version of Ruby and then installing the gems that way. I have found another way install Jekyll via the Ubuntu version of Ruby. This is my preferred way now, as the Linux version of these tools are updated more frequently than the Windows versions.

 

1. Install Ubuntu bash on Windows

  1. Enable Windows subsystem for Linux.
    Follow this short guide on how to enable it https://msdn.microsoft.com/en-us/commandline/wsl/install_guide
    image
  2. After following the steps in the guide above. Simply start the Ubuntu bash shell
    image

 

2. Install Ruby & Jekyll

# Get Ubuntu up to date and install Ruby
sudo apt-get update -y && sudo apt-get upgrade -y
sudo apt-get install -y build-essential ruby-full

# update ruby gems and install Jekyll
sudo gem update –system
sudo gem install jekyll bundler

Then you can do the standard ‘jekyll new foldername` and `jekyll serve` to host it.

image

FYI: the error message in the screenshot about Bash on windows, no longer applies. As the Windows 10 Creators Edition resolved this issue

Reghack “Down Under” event roundup

Reghack was a 3 day hackfest that ran in Melbourne, Australia https://reghack.org/

The problem statement asked:
Do you have an interest in helping solve regulatory issues in the Financial Services and the Energy Sector in Australia?
How do we use RegTech to make regulatory compliance a strategic advantage that’s a win for the regulator, market participants and the consumer?

The focus of the event was to try and invigorate local innovation in the energy & financial services sectors, by allowing people to come together and explore how Blockchain could be utilised in these areas. The event was the brainchild of Chami Akmeemana https://www.linkedin.com/in/chami1/ who is a director of ConsenSys, the largest Blockchain focused consulting company in the world. Chami came to Melbourne and asked for local community support to help him organise and run the event. I was lucky enough to be tapped by Chami and invited to assist. I helped out by providing sponsorship for the meals via Microsoft, and delivered training to help upskill the community beforehand (more about that below).

The event had around 90 participants, with many more volunteers on the day. In the end 14 teams pitched their ideas which ranged from energy trading systems, ways to authenticate documents, to ways to eliminate GST during B2B transactions.
A big thanks to Chami for organising it all, and to all the volunteers that helped make the event a resounding success.

20170512_204517

Note: The roundup of the teams pitches are at the bottom of this post.

Continue reading