Moving up the stack, from hardware to IoT

This post talks about my past few years of work doing hardware consulting and realizing just how many clients are looking for help building IoT projects. I’d like to offer more services to that segment of customers, so I need to take on more skills. In the process, I joined a startup that enables people just like me.

Some of the content of this article was also discussed on my podcast The Amp Hour, on episode 569

When we last left off…

The last time I did a career post on my blog, I wrote about how I would be “Redirecting Beams”. That was really an announcement about how I was going to work for Hologram, a cellular MVNO. I was super excited about getting to work on hardware and interacting directly with developers. I was less enthused when Hologram stopped making embedded hardware.

While I still like and use the cellular service that Hologram provides, I politely declined to continue working with them. After stating I was happiest when I was doing hands-on electronics design, my then-girlfriend thought aloud “Hey, maybe you should try to do that all the time. Isn’t that what consultants do?”. I have been doing hardware consulting for industrial and commercial clients ever since via my company, Analog Life LLC. I have also married that wonderful woman, you don’t let that kind of insight go.

What a wonderful and unexpected gift that has been. Moving into hardware consulting felt similar to my experience early in my career: being overwhelmed with all the things I had to learn and then realizing the patterns and pitfalls of design. I was stronger than I remembered. I could do this. I got out of my head and started to enjoy the process. I continued to find work and clients that needed help getting things out into the world.

Down at the hardware level

I love building electronics. I was lucky enough to find clients with interesting problems where I got to design circuit boards on a regular basis. But I realized I needed a large slate of clients to keep me busy, not just one large client. Why?

I normally refer to hardware work as “choppy”. Unlike past jobs at large corporations, small consulting clients don’t need circuit boards built, tested, debugged, and redesigned over and over again. If I’m doing my work correctly, I am not needed after the beginning of the design cycle. Iteration happens after we wring out all of the bugs, including on the firmware and software.

Moving up to firmware

I like making money and billing hours…so why not try to take on more of the work? I decided to start “moving up the stack”, not just designing hardware, but also offering firmware design services to customers. To ensure I was offering a good product, I started hiring tutors, learning more of the firmware side of things, and offering to take on that part of consulting projects for my clients. Not every project was a good fit, if they had very specific or difficult firmware needs. But some of the firmware projects were simple implementations and I was able to use unbilled time and hired help for tough portions of the code in order to deliver at least a proof-of-concept.

If I really wanted to learn how to do all of the firmware work, I should take on the risk of being the end client. An added benefit is I could talk about every aspect of it on The Amp Hour or on the newly launched Contextual Electronics Podcast. This time lined up with the start of the COVID lockdowns, so I took the leap and started the Advanced BLE-CELL (ABC) board for Contextual Electronics.

I started with what I knew: the hardware. I recorded a video per day on a design that had been in my head, based on client requests. It would be a template for various industrial companies wanting to create low-cost, bespoke cellular projects. I would have not only a cellular modem (BG95), but also a low cost and popular Bluetooth SOC onboard (nRF52840).

6 months later, I had a slate of videos for people to watch of me going through the entire process from concept all the way through writing firmware to get the first blinking LED on board. But…what about connecting this thing to the internet?

Building products, not just boards

A quick aside: A couple of years back, a mechanical engineer sent me a DM about wanting to design a Bluetooth-based widget for his shop. He asked if I could teach him how to build one. I explained that I could teach him many aspects of electronics: how an LED worked, which components to choose, how to layout a circuit board that contained a Bluetooth module or SOC. Building an entire product was out of the scope for Contextual Electronics. He politely declined and moved on to build one with a development shop. As I thought about his request and the required skills, I realized I also required this training: it was out of the scope of things I would be capable of doing on my own. I always assumed I would be working on a team that would have a firmware engineer or that I could hire one for a consulting job.

Since I didn’t yet have the capabilities to build the entire system, I hired someone to help me build the ABC system. I wanted to move the project forward, so I decided to hire someone from Upwork. Not only to develop some of the firmware but to teach me in the process (that was part of the job listing).

I also wanted to learn about Real-Time Operating Systems (RTOS), so I included that in the job requirement. An RTOS would enable networking, more precisely controlled operations on microcontrollers, and simplify scheduling of tasks as more was asked of the device. Asking around, I kept hearing about the Zephyr Project, which was a cross-platform, open-source RTOS. I had another client who was also using Zephyr, so after some preliminary research, I chose that for the ABC board.

I hired a firmware engineer named Bilal to write firmware and show me the ropes of Zephyr. He wrote a driver for the cellular modem on the ABC board in Zephyr that we ended up contributing back to the project (pushing upstream). Bilal was a great teacher and we recorded some videos together about the process.

Moving up to the network (IoT) level

Looking at my consulting projects, I realized almost all of them involved adding connectivity to older projects or creating new projects that also have connectivity. What’s more, I had placed myself the same position some of my clients were in. I have a device, I have hired someone to build me hardware and firmware. What do we do with the data being generated from the device? I started looking for options for connecting things to the internet and handling network traffic.

Once again I would need to learn how to to “move up the stack”. It’s not enough to have firmware and hardware designed to communicate from the ABC board. If I wanted to take on more of this project and future IoT projects, I would have to learn how to write software on the receiving side (the internet). Bilal got the project to the point where it was sending MQTT messages (a messaging protocol) over cellular to a test server, but now what?

It was around this time that I got in touch with a friend who lived in the Bay Area. Jonathan had been working at companies like Particle, Google Nest, WeWork, and we chatted about his new startup idea. I mentioned my work with Bilal on Zephyr and he mentioned how his new company was working on a potential solution. I asked Jonathan to come on The Amp Hour in a few months’ time to talk about the startup. He was still in “stealth” mode, so instead, we chatted more generally about what makes IoT difficult. We were not short on topics. I complained about the hardware aspects and the opaqueness of what happens to data after it leaves the device. He mentioned the structural problems of being locked into cloud providers and developing vertically integrated solutions. Embedded devices like the ABC board require such specific architecture to all work together, and many of the choices that impact an IoT platform a year into development are made the day that a hardware engineer starts the schematic for a product. IoT is indeed difficult, and Jonathan laid out a plan for how to make it better. I agreed there’s a lot of room for improvement and went back to work on my shoddy implementation on the ABC board, since I was now trying to make things work on my own.

Switching to an exciting new product

The ABC board was put aside around this time for a very interesting, very new project: my daughter was a few months out from being born. I needed to prepare my consulting clients for me to take some time off. The timing worked out that a couple of my client projects were winding down anyway, so I was able to slide into a self-funded paternity leave alongside my wife.

As I emerged from the early parenting sleep-deprivation haze, I started thinking about the ABC board and restarting development. How was I going to get this board sending data to the internet? Where do I send that data? Who needs the data? How do I manage devices that I have created and deployed into the field? How do I package this up for future clients and sell the ABC board as a “starting point” for their cellular and Bluetooth designs? So many questions to answer.

Jonathan also got in touch and updated me on the progress of the company (Golioth) as he prepared to announce the availability to the world. I learned more about how it basically was designed to answer many of the questions I had. Golioth is built on top of Zephyr. It provides cloud infrastructure to connect to each of your devices. It has a “console” to manage these devices. It talks to MCUboot (an open-source bootloader) on the device to send updates over the air devices in the field. It has a built-in database function called LightDB so you can send and receive updates from the device.

It felt like Golioth was designed for me. It kind of was. Not me. But hardware engineers. Jonathan asked if I wanted to do some consulting around developer relations, helping support engineers understand the value of the platform and how to use it. So I have been doing that since June of 2021, making videos and onboarding new users to the platform. I enjoyed working with the team so much, I decided to join officially as of November 2021, so I can be part of the company long-term.

Moving up the stack, from hardware to the cloud

I have successfully been “moving up the stack” and doing hardware, firmware, and cloud, using Golioth. Towards the beginning of my consulting for Golioth, I also met a small software team looking to build an IoT product, so I recommended trying Golioth for fast prototyping and easy scaling later. I was able to create a proof of concept design using an off-the-shelf cellular modem, some external electronics, and some of the samples that Golioth offers. This prototype could:

• Communicate over cellular networks
• Had firmware on the microcontroller that controls/reads
  • Solenoids
  • LEDs
  • Reed switches
• Had full access to push and pull data to the device from a web terminal (which could also be accessed via a REST API)
• Had over-the-air firmware update capability

And all of that took 3 hours to build.

I’m not saying I have solved creating any random IoT device. But I am much more capable as a result of this new platform. Now that I’m working with Golioth, I’m going to build my muscles around building fast turn-around prototypes and securely connecting things to the internet. One of my good friends is joining the team and we’re going to work together to build interesting devices.

I still have a lot to learn, especially around Zephyr and the network side of things. But I’m excited to keep building lots of interesting hardware in the future. Check out the Golioth YouTube channel and blog to see what I have been building and talking about lately.