How I fell in love with test-driven development Part #2

Part 1 of this article is here , and in that part we covered the basics of Test-Driven Development (TDD) and focused on why we don’t need to double the effort for implementing TDD, even if we need to write tests first before adding new features.

The key thing is that the constant stability and reliability of TDD code pays off really well at the end of the day. Now, let's talk about how to write relevant tests in an embedded project and how it is even possible to do embedded development with TDD.

Let’s Get Portable

First of all, to do TDD, the code needs to be portable because you are going to use it on two different systems at least. It has to run on the target system obviously, and on the embedded hardware. The other important target is your development system, for example your desktop computer or notebook. By using these 2 different systems, one is able to speed up development and debugging. Updating a firmware is often more complicated than hitting the enter on your keyboard, sometimes there are push buttons or jumpers on the target system for turning the device into firmware update mode, or a power reset is needed. It does not matter if you have an old school serial link, or some sophisticated USB device for programming, transferring the firmware could be painfully slow. Testing on the target is also slow and often needs some physical involvement so you need to release your keyboard. It makes sense to touch the target system as little as possible. Most of the coding should be done on the development system.

Running and developing embedded code on a desktop computer has been a well known practice for a very long time. The benefit is to speed things up. Updating a firmware is often more complicated than hitting the enter key on your keyboard. Sometimes there are push buttons or jumpers on the target for turning the device into firmware update mode, or a power reset is needed. It does not matter if you have an old school serial link or some sophisticated USB device for programming, transferring the firmware could be painfully slow. Testing on the target is also slow and often needs some physical involvement, so you need to release your keyboard. It makes sense to touch the target system as little as possible.

Even if you are not willing to use TDD, the need of running the code on your desktop computer comes naturally. The most common thing to do is running and testing the business logic on the desktop. In general, C / C++ languages are considered as a standard, and you are able to write portable codes. So you have the option to run your code on a 64 bit powerful modern computer and on your 8/16/32 bit target with very few resources, kilobytes of RAM and flash storage instead of gigabytes. But you cannot take it for granted; you have to work carefully to achieve it.

There are several resources out there about how to make your C / C++ code portable. You have to take care about Endianness (the order in which computer memory stores a sequence of bytes), size of data types, signed or unsigned chars, standard library calls and so on. Embedded systems always use some platform specific calls for accessing their peripherals, you have to separate them all. When you first take a look at an embedded source code, this is the most visible quality measure. Is the platform specific code well separated? Good products can outlive the availability of their electronic parts. Sometimes we need to migrate the firmware to a different architecture and other times it is all about profitability. There is a new and cheap microcontroller available, and depending on the production volume, we should migrate. When it happens, no one likes to modify every single source file because of the bad practice of not having the platform dependent parts separated. The more source files are needed to modify, the more additional testing and bugs you are facing.

Advantages of TDD

Let’s assume you have a well defined and small interface between the platform dependent and independent codes. You are not going to put any business logic to the dependent part, and you are very strict to keep the independent part portable - this is the first important step in order to do TDD.

What’s next? How do you run the firmware on your personal computer which does not have the peripherals the embedded hardware it’s made for? For example, I will use my hot beverage vending machine. The business logic is taking care of water heating, motors for moving powders, motors for mixing water with powder, water switches, water level and temperature sensors, grinder, scale and espresso motor.

At first glance, you can say that this is an easy development project and there is no need to use heavy weapons like TDD. Many years ago, during developing our first vending machine, we used to joke around with the client: “Hey, no worries, we just switch on and off some relays, and the coffee is ready, a few weeks of coding, tops”. Well, making a good coffee on a prototype vending machine is quite easily doable; but when it comes to having a fully flexible, remote controllable device in production, installed on high traffic locations with minimal need of maintenance - that's a different problem.

Before TDD, the old way of implementing a new feature was 15-30 minutes of coding, then a full hour of manual testing by wasting a lot of coffee. Even if the feature seemed to be working well, we always had to do manual testing constantly in parallel with development. Sometimes we needed a few hours of testing to catch or reproduce a single bug because of the physics of heating the water to a specific temperature, doing something while waiting for it to cool down and then doing it over and over 50 times until the bug appeared – you can imagine how painful it would be? Coding continuously and effectively in a flow is a non-existing thing in this environment.

A New Order

Ok, let’s write a new feature of filling in the water tank of the vending machine. By using TDD, we need the test first: switch on the water, wait for a specific time, check if the tank is full, see the test failing. This is such an unusual task for a desktop computer without a water tank! The key thing is to simulate every physical event to an optimal complexity. This is the time when our interface between the platform dependent and independent parts comes into play. Normally, switching on the water looks like this:

It means we activate a general purpose IO port of the microcontroller, which is connected to a relay or any semiconductor switch on the Printed Circuit Board. Please note that the gpio_set() function is not supposed to be a library function. It has to be your own interface function, otherwise your code wouldn’t be portable. Any time you change the platform, you need to modify this. Two implementations are needed, one for the target which does GPIO on the actual embedded platform, and one for simulation. The optimal complexity of a simulation model here means as simple as possible. If the water is ON, we could run a timer which increases the virtual water level from 0 to 100%. At the same time we decrease the level as any water switch for powders or expresso is open. It does not need to be very precise, because the real water flow isn’t precise either, depending on the input water pressure. We can take a look at the real timing and set the virtual speed accordingly.

Some More Complexity Please

Let’s see an example of a medium complexity simulation model. Precise control of the espresso water temperature is essential. The sweet spot is around 92 Celsius, going below and above are also problematic from a quality point of view. Some espresso modules have a relatively small water capacity, while the heater is powerful, because we don’t want to make the customer wait. The heating algorithm is not “heat until it is warm enough” at all. That would cause a significant overshoot, which affects not only the actual cup of espresso but the next cup of instant beverage too. For simulation, we need a thermal model to handle this, to be able to set up a proper thermal control. Even some factory firmwares of big companies aren’t able to handle this properly! They have a simple heating algorithm, which works fine when the vending machine is warm, but totally fails at the beginning of the day. Experienced customers wouldn’t drink the first espresso of the day.

By having all simulation functions implemented, it is possible to “make hot beverages” on the desktop computer with very similar timing to the real hardware. Well, time is relative, instead of waiting 25 seconds for a combined espresso + instant beverage (i.e. coffee with milk and hazelnut), the simulator can run it in milliseconds. It quickly eliminates 90% of manual testing, no need to waste a lot of time and coffee material for testing new features. Now, nothing prevents us from working fast and efficiently, in flow the same way we do in a pure programming project without having complicated and slow hardware.

TDD Summary

Even if an embedded project is more likely on-site and hard to do remotely, I am quite fine working on vending machines fully remote since I can always watch the serial console of the vending machine online to see what happens and the client even leave a phone in conference mode to make sure I can watch and listen to the machine. But thanks to the portable code and TDD, this kind of supervision is less frequently needed and the majority of the development effort happens without bothering the real machine. Don’t get me wrong, I love to create real pieces of hardware and play around with them, but similar to corporate meetings, from an efficiency point of view, it makes sense not to have too many of them.

When you build all your projects to be testable, at some point unexpected miracles could happen. The vending machine mentioned above has a separate WiFi communication module, different hardware, a different software stack, and still, they are able to communicate with each other in the simulator. I was really surprised to see how a simulated vending machine could log into the production Python server on the internet by using the simulated WiFi module. It’s quite similar to The Matrix movie, right?

I hope I was able to give you some inspiration on giving a chance to TDD, if you haven’t tried it yet. Do your own miracles!