Prettify your Windows terminal

I do my personal coding on a Macbook Pro running macOS. My favourite terminal application is iTerm2, and I use it daily when working on open source software. You can tweak the appearance and behaviour of iTerm2's console really easily and in a granular fashion.

I also own a Windows desktop machine, which I have around to test my own software on. It took me a while to get a pretty and cohesive experience set up for an enjoyable experience on Windows, so I am sharing my setup with you in case you would also like a similar aesthetic.

First things first - there is a rad NodeJS terminal app written in Electron called hyper that you can install and will require less tweaking than my setup. I chose not to run hyper because my machine does not have super impressive hardware specs. The machine runs too slowly to support an Electron app in addition to my web browser and code editor, because Electron apps are a Chrome browser and NodeJS process in one running package. Running a native terminal application makes a bit more sense for my use case.

Here is what the final result looks like:

The whole desktop (view image in a new tab to zoom):

pic of the screenshot of consolez

If you'd like a similar look, here are the steps involved:

1. Install Github Desktop

The most efficient way to get a 'unix-like' experience on Windows is to install git bash via Github Desktop. Using git bash is a big win for unix-like shell command consistency, and you'll have git automatically installed and working out of the box for you.

Download Github Desktop for Windows and install it. When you run it for the first time, it'll prompt you to log in with your Github account. Once you're all logged in, leave Github Desktop open for later.

2. Configure your prompt

You can now run the Shell program included with Github Desktop by finding it in your Start menu under the Github directory. The terminal that opens is running git-bash, and you should find git working out of the box, as well as the typical unix commands such as ls, cat etc.

if you'd like to tweak your shell prompt, you should be able to find it by looking in your user directory's AppData/Local files. The full path will probably look something like this:

C:\Users\<your username>\AppData\Local\GitHub\PortableGit_1234…\etc\profile.d\git-prompt.sh

You can reference / use my shell prompt to get started, if you like! It is below.

But the terminal still doesn't look super pretty yet, huh? Not to worry! Let's use consoleZ to tweak the appearance a little further!

3. Install and configure consoleZ

consoleZ is a wrapper application for your terminal of choice. It unlocks some cool appearance configurations that you won't get by running git-bash or Powershell alone. It's the best one I've found to have control over fonts, colours, background transparency, etc.

Install consoleZ by following the instructions in the consoleZ repo's README.

Once you boot it up, open the settings by hitting Ctrl + S, and you'll see lots of settings you can tweak to your heart's content. Before you do that though, we need to tell consoleZ to run our git-bash shell that Github Desktop installed for us. Under the 'Console' pane of the settings, in the 'Shell' input field, type bash.exe —login -i.

Once you save your settings, you might notice that consoleZ pops up an error and doesn't work anymore! Not to worry, we're going to fix that in the next step!

4. Point Github Shell at consoleZ

Switch back to Github Desktop. Open up the settings pane by clicking the little gear on the top right of the Github Desktop GUI. Within the 'Default shell' section, choose 'Custom' and fill in the input underneath with the path to where you installed consoleZ (it might be different to my example below):

pic of github desktop settings

5. Start up Github Shell

In order to start up your terminal from now on, you'll need to run Github Shell. You have a couple of options to do this. You'll find it in your Start menu under Github, you can pin it to your tiles in your Start menu as a shortcut, or you can launch Github Shell from the Github Desktop application.

If all has been set up correctly, you should see consoleZ start up, with git-bash running inside! Yay! From here, you can access the settings and tweak the appearance and other properties to your heart's content.

I am using a colour theme called 'space gray'. If you'd like to reference or start with my configuration file instead of manually doing it in the settings, go for it! You should be able to find your consoleZ settings by looking for a console.xml file in your user's AppData/Roaming directory. My path looks like this:

C:\Users\<your username>\AppData\Roaming\Console\console.xml

Enjoy your pretty terminal! If you follow these steps and they work out for you, I'd love to see you share a screenshot in the comments!

Wireless 3D printing station

pic of raspi and octoprint

When I first got into 3D printing, I'd been outsourcing all of my printing needs to Shapeways. I was working on a brand new jewellery range at the time, and was prototyping different forms. I wouldn't call that rapid prototyping though, given the wait on even plastics used to be two weeks at the time. They turn things around a lot faster these days, though!

A year or two later I moved to a city which was about to open a hackerspace in the downtown area. Once it opened, three 3D printers were available to use, which stepped up my access to printing things that I needed. This was a pretty big deal, because I got to learn how to use the machines. The process was no longer abstracted away. I could turn around things much faster as well if I needed to.

I remember when I then bought my first 3D printer kit for my home. It was deliriously exciting to think that I could print any time I wanted, and tweak a myriad of settings to perfect the output without annoying others.

The 3D printing workstation came about in two stages: convenience first, and then later, space.

I lived by myself in a rather generously sized, stupidly affordable two bedroom condo at the time. The printer took pride of place in my rather large study. This study easily held two work tables and a rather bulky writing desk. I had a sewing station, a computer station for my coding needs, and a dedicated place for my new 3D printer as well.

It became apparent rather quickly though, that I was constantly tethered to the printer in some way. I started by printing via a USB connection to my computer. This was a rather terrifying endeavour. One tiny interruption would be catastrophic for your currently running print job. Say, you're four hours into a five hour print, going strong. You forget you're tethered and click 'yes' to a software update, which you didn't notice would require a restart of your computer. Welp, you just halted the print, and in most cases this is not recoverable. Perhaps your friend would like to Skype with you, and you have to turn her down because you're stuck next to a bunch of noisy stepper motors on a vibrating printer for the next two hours. You get my drift!

I also tried printing from an SD card, however I found that I was continually running back and forth between the computer and the printer, reloading and starting up a gcode file that had been tweaked in some minute way for better settings. It was better, but not optimal.

I started looking into other options online, and stumbled upon Octoprint. It was written by Gina Häußge, and BQ recently sponsored work and maintenance on it going forward. Octoprint is a wonderful python based service that can run on any computer attached to a 3D printer. It serves up a web app, meaning you can access this web app simple by visiting the IP address of the computer you have it running on. You can upload gcode files, run gcode commands on the printer, and start print jobs all from the web interface.

I took the path that most have, and decided to run Octoprint from a Raspberry Pi. The Raspberry Pi is ideal, because it's a very small computer capable of running Octoprint just fine (in fact I'm still using my old Pi model B!). It doesn't take up to much desk space either which is really nice. Having a little machine dedicated to running the 3D printer has really made things so much more convenient.

Getting up and running on Octoprint is pretty straightforward. You can install it yourself on a Raspberry Pi, or you can download a pre-made Raspian image with Octoprint ready to go made by Guy Sheffer called OctoPi, which is even easier.

Here is a screencast I recorded of my old printer running on Octoprint. Octoprint supports webcam streaming and timelapse recording which is an awesome feature you'll notice in the video below:

And an example of a timelapse from Octoprint, also on my old printer:

Pretty neat, huh!

Fast forward to last year. New year, new printer (!!), new city, and new apartment! The new city/apartment caused a fresh problem. This was New York City. I am sharing a rather tiny two bedroom apartment with my partner. I downsized a LOT to make this happen. I got rid of my old writing desk, and my partner uses one of the work tables as the desk he works from every day (he works from home).

When we bought a new, larger printer, we had to place it straddling our two small work tables. This ate into our desktop space, was really noisy, and it shook the two tables as it printed. Not ideal!

Given the limited space we have in the study (second bedroom), I window shopped online until I found the Stuva Storage Combination from Ikea.

pic of stuva from ikea website

I picked it up for $135 plus additional for delivery (NYC = I got rid of my car). What sold the Stuva for me was how narrow it is (we had managed to create only a tiny space for the new printer location), and how much storage it had. Finally, we'd be able to unpack the rolls of filament from the cardboard boxes in the closet!

What we really wanted was for the printer to sit on top of the workstation and nothing else. The Raspberry Pi, the wifi extender, and the power supply should be hidden away. We installed the shelf inside the Stuva in a high position (after measuring the height of the power supply) in order to achieve this.

This is the final result, squished into our study right next to the entrance:

pic of the station1

When we're worried about draughts (rarely, because we mostly print in PLA) we just close the door, and open the window on the other side of the room to prevent overheating / provide ventilation.

The wireless setup inside:

pic of the station2

Bottom drawer, full of filament:

pic of the station3

It looks beautiful, frees up desk space, and allows storage of filament, kapton tape, tools, etc right with the printer. Win win!

I'd love to see other printer setups, feel free to comment with photos below!

Using Johnny-Five within an Electron app

Johnny-Five is a pretty fabulous NodeJS based robotics library. You can use it to create all sorts of things!

Running your Johnny-Five application within Electron is a great way to add a user interface to your robotics. It does require a couple of additional steps to get it up and running correctly, however, so I have outlined them below.

  1. Serialport will probably fail to be found by Electron's Node process. We can fix it! Follow the steps in a previous post of mine to get this going, making sure the rebuild and rename steps are performed on the serialport directory within ./node_modules/johnny-five/node_modules/ instead of a fresh serialport install.

  2. Johnny-Five uses process.stdin, which is not available for Electron's use. This causes Electron's render process to crash! We can reroute process.stdin to a compatible replacement stream though, to get around this. Before you do any of your regular Johnny-Five code, paste the following into your JavaScript file you're using within Electron:

var Readable = require("stream").Readable;  
var util = require("util");  
util.inherits(MyStream, Readable);  
function MyStream(opt) {  
  Readable.call(this, opt);
}
MyStream.prototype._read = function() {};  
// hook in our stream
process.__defineGetter__("stdin", function() {  
  if (process.__stdin) return process.__stdin;
  process.__stdin = new MyStream();
  return process.__stdin;
});

// then Johnny-Five code goes below here!

Thanks to Rick Waldron for this last step, which he wrote up for a different use case (NW.js) but it works great for Electron too. I just copied the code straight from the before mentioned link.

Let there be light! You should now be able to run your Electron app as normal and blink that LED!

yellow johnny-five logo

The avrgirl project: manual testing

green baseball cap with the letters 'AVR' written on it

Publishing your code for others to use is a scary, but often rewarding accomplishment. It can be really exciting to witness peers and strangers incorporating a library of yours into their own projects.

Being an open source maintainer brings a lot of responsibilities to the table. Managing issues, pull requests, and emergency bug fixes can be a lot for one person to deal with. If your project cannot rely solely on automated testing, this can be a real burden. This is particularly true in the hardware library scene. I have experienced all sorts of weird bugs pop up when the same code worked the day before. Things like operating system changes and runtime upgrades can expose the fragility of how some hardware libraries work, more readily then something, say, a web based library.

I have run into this recently when publishing and maintaining libraries falling under the AVRGirl project, which I started working on in early June last year. While I have learned a lot during the process, there is still a lot more for me to know. These gaps in knowledge often manifest into slightly different experiences across users' operating systems and versions of NodeJS, which is what all AVRGirl packages are written in. When I say 'experiences', what I really mean is bugs, of course ;)

When I write unit tests for AVRGirl, the most notorious parts of working with hardware get stubbed out. This is stuff like serial interfaces, and responses from hardware are faked. This tends to be where a lot of things go wrong, and where compatibility issues rear their ugly head. In other words, my code needs to know about all of the edge cases and fragilities, and provide for them. Simulating this kind of stuff in unit tests is not at all practical and introduces some tech debt and readability issues for potential contributors. It also doesn't help me stay on top of OS and NodeJS version changes, and their effect on my libraries/code.

The problem

So let's break down exactly what all this means. We'll focus on one library in particular, which is AVRGirl Arduino. AVRGirl Arduino flashes precompiled code to a myriad of supported Arduino boards. What does this look like, in testing terms?

  1. 16 supported boards
  2. OSX, Windows, Linux (and all modern releases/versions within)
  3. NodeJS versions 0.10, 0.12, 4.x, and 5.x (currently alpha testing 5.x so we won't count it)

This works out to be approximately 144 integration test cases, if we're just counting operating systems as giant buckets, and not splitting them out to things like OSX Yosemite, El Capitan etc etc.

What I'm getting at here, is that I cannot sit down and manually test everything alone for both releases and environment changes/additions. Well, that and hold down a full time job and occasionally not code from time to time :P We all need time away.

AVRGirl Arduino was picking up a bit of interest a couple of months after releasing the first version, so I decided to reach out to potential testers by opening the following issue.

screencap of issue #4 under avrgirl-arduino github repository

It was a quick explanation of what I wanted and why. I added the 'help wanted' label and thought that would be it. After mostly radio silence, I started thinking long and hard about why I've had no shortage of developers opening pull requests with new features and bug fixes, but not a lot of feedback around if the library actually works as expected in these 144 different combinations. No one was chomping at the bit to sit down and plug in a bunch of boards and run some quick set up code.

Wait, but that was just it! I had not provided an easy way to manually test AVRGirl Arduino. Sure, I'd written some vague instructions in a Github issue, but that's not a very good experience for people wanting to help. I identified a few things that would need to be better if I was going to ask others to manually test for me:

  1. Manual testing is boring for most people
  2. You don't get a 'little green square' contribution reward for manually testing
  3. People who are new to hardware probably had no idea where to start even if they wanted to help. My issue made a lot of assumptions about others' knowledge, which is not inclusive.
  4. There were too many steps involved and too many bridges to cross in order for someone to invest in helping. People's time is valuable and I was not respecting that.

A proposed solution

I decided to write a small app to run within AVRGirl Arduino, its only job being to allow an easy way to manually test AVRGirl Arduino. Embedding it within the library itself allows for great portability, and no additional installation of anything else needed should someone like to help test. The downside is that it introduces more dependencies to AVRGirl, which carries more risk for issues. I tried to keep it relatively lightweight to mitigate this.

I called it AVRGirl Test Pilot, and connected it to the CLI tool within AVRGirl Arduino. The 2 steps required to start testing are:

  1. With NodeJS installed, run npm install -g avrgirl-arduino
  2. Run avrgirl-arduino test-pilot

That's it! A local server will spin up and a browser window will open with a prompt to log in via their Github account:

screenshot of login screen

Once logged in, the user will see the following steps to follow:

screenshot of 3 steps

The test process simply uploads some code to their Arduino and verifies it worked. The resulting report is automatically generated for them to submit:

screenshot of report screen

Submitting that form will send the report details to a remote database that I host. The screenshot above shows what details I'm after when the test is complete:

  1. The Arduino board they were testing with
  2. NodeJS version they're using
  3. Operating system and version
  4. Version of AVRGirl Arduino
  5. Version of Test Pilot
  6. Who the tester is (optional)
  7. Additional notes (optional)
  8. Any errors that might have been returned

This info is enormously useful for me. It gives me a place to start, and doesn't require the tester to go in and open an issue on the Github repository (where I would then begin to ask them a million questions about what software and hardware they were using etc).

In time when I finish setting up the Github organisation for AVRGirl, if the testers opted in, they'll be invited to be part of the awesome testers team within the organisation. This gives them a little 'badge' for their Github profile, which I hope will be seen as a nice little gesture to say 'thank you'.

So far, I've had two bugs come in from submitted reports, which had already made the efforts behind Test Pilot worth it. I'm especially curious to see if Test Pilot will help hardware beginners to feel less intimidated helping out with a project such as AVRGirl.

If you'd like to be a test pilot for AVRGirl Arduino, check out the 2 steps mentioned earlier in this blog post, or find help in the README of the AVRGirl Arduino repository on Github. And thank you so much in advance! <3

Using node-serialport in an electron app

So you want to create some serialport magic within an electron app? Right on, that sounds like you're creating something pretty amazing for the desktop!

In the past this scenario has been a pretty tough feat to achieve, however with the convergence of NodeJS and iojs into NodeJS 4.x, the road to serialport fun for desktop apps is a much shorter and more enjoyable one.

The following instructions are designed with Linux and OSX operating systems in mind.

Important note: if you have installed NodeJS version 4.x or greater, and are using at least electron-prebuilt version 34.0.1, you may not need all of the steps below anymore. In a lot of cases, simply running npm install serialport within your electron application directory, and requiring/using serialport as you usually do should work just fine!

Let's do this.

Steps for success

Make sure you're on NodeJS version 4.0 or higher.

I'm going to assume you have already installed electron-prebuilt for your machine. If not, run:

npm install -g electron-prebuilt  

Next, set up your electron app boilerplate if you haven't already. This is essentially creating the following files in an empty directory:

  • package.json
  • main.js
  • index.html

See the electron quick start docs for how to set these files up.

Ok! Now let's install serialport. Run this from the root directory of your app:

npm install --save serialport  

You'll notice that a node_modules dir will appear, with serialport within. So far so good. At this point, you could try requiring serialport within your electron's index.html file and see if it works as expected. If it doesn't, never fear, let's try the additional steps below!

We gotta rebuild serialport to work with electron. To do this, we need to install electron-rebuild:

npm install --save-dev electron-rebuild  

The last step is to run electron-rebuild, once again, just in the root directory of the app:

./node_modules/.bin/electron-rebuild -$(electron -v)

A quick test dropped into index.html would look like this:

<script>  
  var sp = require('serialport');

  sp.list(function(err, ports) {
    console.log(ports);
  });
</script>  

Woo! We can now run electron and start up our app:

electron .  

Uh oh! Do you see the following error in your electron app's console?

Uncaught Error: Cannot find module '~/my-electron-app/node_modules/serialport/build/serialport/v2.0.2/Release/node-v46-darwin-x64/serialport.node'

This be an unfortunate issue. But this is pretty easy to fix for now :)

If you rename the directory within Release to what the console output is expecting, you're right as rain and it'll find the serialport.node file correctly. Have a look in the Release directory to see what the child directory was named instead, and rename it.

Example:

mv ~/my-electron-app/node_modules/serialport/build/serialport/v2.0.2/Release/electron-v34.0.1-darwin-x64 ~/my-electron-app/node_modules/serialport/build/serialport/v2.0.2/Release/node-v46-darwin-x64  

Try running your electron app again, or refresh it with Cmd/Ctrl + R. You should see a bunch of ports listed within an array in your console this time! Sweeeeeet.

Looks like @stevekinney and I are no longer NodeJS desktop app frenemies.

Have fun!

The avrgirl project: supporting STK500v2

This is the third post in a series about avrgirl.
The first is avrgirl project: an introduction.
The second is avrgirl project: supporting Arduino.

When programming Atmel® AVR microchips, there are several communication protocols available, and it's important to choose the correct one. Adhering to the correct protocol is essential in order for your programmer of choice to understand what you want it to do.

So far, avrgirl has had to support three of these protocols: STK500v1, STK500v2, and AVR109. In this post, we'll go into detail for one of these protocols in particular: STK500v2 and the resulting avrgirl release: avrgirl-stk500v2.

Even though there is already a package available in npm for STK500v2 (thank you Ryan), I thought it would be beneficial to write an alternative version for avrgirl, as I wanted the API to look and work quite different. Not to mention that I was pretty sure I'd learn a lot from the process of writing it from scratch using the datasheet. Read it if you're interested in seeing what kind of resources I work with when developing software for general electronics!

avrgirl-stk500v2 implements a protocol only, but it still works as a standalone package to use with any USB device which speaks STK500v2. This was by design. I tried to keep it as generic as possible.

The basics

STK500v2 is quite a verbose protocol compared to others, even STK500v1. Headers and checksums are required even to send a simple request or instruction.

Here is the general communication signature for most instructions:

Message Format
MESSAGE START
SEQUENCE NUMBER
MESSAGE LENGTH (MSB)
MESSAGE LENGTH (LSB)
TOKEN
MESSAGE BODY
CHECKSUM

Responses from the programmer follow an identical signature.

Message start

As the name suggests, this byte signifies the start of a new message. The byte value is the same each and every time - it's 0x1B.

Sequence number

This number needs to be incremented by 1 for each message sent. Once the sequence value reaches 255, it should wrap around back to 0 again.

Message length

This tells the programmer how many bytes are in the main message body you're sending. It does not count the header bytes, nor the checksum. We supply the length in both MSB (most significant byte first) and LSB (least significant byte first) formats.

Token

Always 0x0E.

Message body

Here is where you put your instructions for the microchip. It could be asking for its signature, writing a fuse, or a number of other things that the microchip supports.

The simplest example of a message body is getting a parameter value from the programmer settings: 0x03, 0x91 - the first byte is the 'get parameter' command, and the second byte is specifically asking for the major version number of the software running on the programmer.

Checksum

This is a byte that is formed with logical shifts of all of the previous bytes in the message, including the headers. This is how we signify that our message is complete.

Variations

There are, as always, exceptions to the rule. Most programmers which use STK500v2 follow the protocol exactly. However, I have found at least one programmer (made by Atmel® themselves, gah) that does not.

The AVRISP mkII, for example, wants to skip both the headers and the checksum at the end. I only found this out through trial and error after some relative frustration. It was technically in the datasheet for the programmer, but I made the wrong assumption that the author had dropped these bytes from each example for the sake of brevity.

As a side effect, this brought in the notion of 'framed' and 'frameless' messages. If a message is framed, the message is wrapped in the headers and the checksum described in the previous section of this post. If it is frameless, only the message body is sent.

The end result that I implemented - requesting frameless mode is an optional parameter in the options object passed into the module when instantiated. The library takes care of the rest.

Communication

I decided early on that avrgirl would work with only more modern programmers which use USB for communication. To offer the best support possible, using either serialport or usb packages will work with avrgirl-stk500v2.

Summing up

avrgirl-stk500v2 is published on npm!

Currently available functionality:

  • Enter / leave programming mode
  • Read programmer / chip signatures
  • Write to EEPROM and Flash memory
  • Read from EEPROM and Flash memory
  • Erase chip memory
  • Read and write fuses
  • Get and set parameters

This list isn't every single thing you can do with this protocol (for example, it doesn't currently support reading locks), but the above methods are what you'd most commonly use.

If you'd like to test out avrgirl-stk500v2, please do! So far I have only had the opportunity to test it on two devices - the Arduino Mega, and the AVRISP mkII. I'd love your feedback, and feel free to open an issue if needed.