Successful Endeavours - Electronics Designs That Work!

Embedded Software


Smart Cities

This follows on from our look at Smart Cities and the technology mix being considered for how you implement them. For this post we will look at the development of a Smart City Telemetry sensor suite and the ICT communications that go with it. This is also a classic IoT case study.

I also want to point out that a Smart World will only happen if we have Smart Regions, Smart Countries, Smart States or Territories, Smart Cities and Smart Neighbourhoods.

arcHUB

My thanks go to The Active Reactor Company for giving me permission to share their story about the development of the arcHUB Telemetry sensor suite which is aimed at the Smart Cities programs as well as being more widely deployable.

arcHub Telemetry Module

arcHub Telemetry Module Logo

A few days ago I had the opportunity to speak with Daniel Mulino who is the State Member for Eastern Victoria. The picture below comes from his visit to our office in Narre Warren. The original post he made along with my explanation is here. I’m giving a more detailed explanation below including some history.

Ray Keefe - arcHUB - Daniel Mulino

Ray Keefe – arcHUB – Daniel Mulino

For those wondering about the device I am holding, it is an arcHUB Smart Cities Telemetry module aimed at Smart Cities projects and environmental monitoring where you don’t have access to, or want the cost of, connecting up mains power. This is designed for The Active Reactor Company and is already involved in 1 Smart Cities deployment and multiple trials of low cost sensor modules by councils and government agencies in 3 states. I can’t yet provide specific details on those as they are covered by non-disclosure agreements.

To understand how we got here, it helps to know the history.

The Active Reactor Company make a product called The Active Reactor. It improves both the efficiency and the life of arc lamps such as low pressure sodium street lights, high pressure Sodium  and metal halide lamps.

The Active Reactor

The Active Reactor

With the advent of LED street lighting their current product is not needed for new installations and so they wanted to secure the future of the business. So a great example of addressing an issue that will arise in the future so you are ready for it rather than just reacting to it once it happens.

Initially the new product was aimed at monitoring LED street lights. One of the big issues with LED lighting is that the LEDs either fail over time or they fade and lose brightness. Or a mixture of both. The fading is a result thermal diffusion in the semiconductor substrate. When they fall by more than 30% then you have to address that as they no longer comply with legal standards for lighting levels. The other catch is that the claimed life of 10+ years isn’t yet proven and so it is expected that there will be many lights that fail early or fade early or both.

Of course, once you have a communicating device that can monitor one thing and report it, it can also monitor other things and report them as well. Plus there were issues with being allowed to monitor the light. And where would the power come? Their inquiries with authorities responsible for the poles would not give permission to tap the power in the pole or light.

So this set us the follow set of constraints to work within:

  • must be battery operated
  • easy to install
  • low cost to make and also run
  • communicate using the cheapest data transport
  • monitor the LED light at night and keep track of the brightness trend
  • send an alert when it is persistently out of specification
  • field life to match the street light (10+ years)

As The Active Reactor Company talked to target users (initially the same people who buy their current product) and got an idea of what they wanted, a very different picture emerged. The people who cared about LED street lighting, also cared about micro climates, and soil moisture levels, and air quality, and foot traffic, and …

So that lead to a change of direction and a look at what else was required. The result is a device aimed at the Smart Cities market that also suits a wide range of other end customers and has the following features you won’t find combined together in conventional devices:

  • battery operated (either solar charged or primary cells)
  • minimum 2 year battery life for standard AA cell alkaline batteries
  • 10+ day running time if solar charging is lost
  • up to 20 days on board non-volatile storage
  • compact form factor
  • multiple sensor types per node (up to 20)
  • sensor area network to minimise data costs
  • over the air firmware upgrades
  • over the air configuration updates
  • variable sample rates and upload timing
  • still has to be low cost to make and also run
  • easy to install

So here is the range of sensors already trialed:

  • wind speed (external anemometer attached)
  • sunlight level
  • night light level (street light monitoring etc)
  • temperature
  • PM2.5 particulate levels
  • PM10 particulate levels
  • Gasses – CO, H2S, SO2, NO2, H2S
  • Humidity
  • People counting (PIR based anonymous counting)
  • Soil moisture levels (external probe)

It is also the HUB and coordinator of a Sensor Area Network that can include modules that can measure any of the above as well as:

  • vibration
  • shock
  • movement
  • water level
  • GPS location
  • USB charger current (for usage analysis)
  • counting any device or system that has a pulse output
  • analog voltage measurements (AC and DC)
arcHUB trial at Fitzroy Gardens

arcHUB trial at Fitzroy Gardens

The arcHUB is solar powered and includes a cellular modem to allow reporting back to a web service. It is designed to mount to a pole using straps but can easily be mounted to a wall or any other typical structure. A typical scenario is measurements every 15 minutes (except people or pulse counting which are continuous) and uploading to the web service every hour.

With the release of CAT-M1 services across Australia by Telstra, we are expecting migrate to this communications standard because it will reduce power consumption by at least a factor of 4 which will further improve battery life.

Quectel BG96 CAT-M1 Module

Quectel BG96 CAT-M1 Module

The arcHUB Peripheral Modules connect via 915MHz ISM Band communications and use standard AA batteries. They can run for between 2 and 5 years depending on what sensors are attached and how often they are read and reported. If you used primary lithium cells then you can expect life beyond 10 years.

The arcHUB Peripheral Modules are also capable of stand alone operation with the addition of an internally fitted cellular modem so you can have a portable people counter module that can be easily moved to a new location and doesn’t require an electrician to install it.

And pretty exciting to also announce that this is not only a designed in Australia product range, but it is also a made in Australia product range.

Again, my thanks to The Active Reactor Company for permission to share this story and if you want to know more, leave a comment and I will put you in touch with them.

Successful Endeavours specialise in Electronics Design and Embedded Software Development, focusing on products that are intended to be Made In AustraliaRay Keefe has developed market leading electronics products in Australia for more than 30 years. This post is Copyright © 2017 Successful Endeavours Pty Ltd.

Software Costing

There is an old saying that goes something like this: “hardware is almost free and comes from China; but software is actually free and comes from India”. Actually not such on old saying, and certainly not true. But we do see signs of this myth being alive and well when providing project pricing and estimates for new clients. I covered some of this in Software Estimation.

Software Estimation

Software Estimation

This was about how to try and work out a Software Development Budget in advance. Including forgetting that the entire Software Development Process involves more than just typing. So is it possible to know what it really  costs from real world (non-imaginary) data?

Software Cost

The answer is that it is. My thanks got to VDC Research who recently did a survey of Embedded Developers and made the data available to subscribers of The Embedded Muse, a software development newsletter authored by Jack Ganssle. If you develop software, especially for Embedded Systems, I recommend you sign up if you aren’t already a subscriber.

Jack Ganssle

Jack Ganssle

Here is a summary of some statistics that gave me insights into real Software Development Costs.

Average Median
Project Team Members 19 7
Project Cost $27,000,000 $250,000
Lines of Code 627,000 20,000

So that is a big spread. Our projects are often below the median level shown here so I was interested to work out what these statistics translate to in cost. The $ are all USD$. And the large lines of code average probably represents larger projects using a major Operating System such as Linux as part of the project.

Cost per team member Cost per line of code
Average $1,421,052 $43.06
Median $35,714 $12.50

My first thought is that we don’t charge enough if these are industry typical figures. A bit more thinking shows the process costs of much larger systems. As far as I know there would be few software developers actually getting $1M for their part in the project. And there will be tools costs also included. The statistic missing for me was the duration the money was spent over. We typically budget $5 per line of code for larger projects (20K lines is a decent sized project for a small embedded system) and $2.50 for smaller ones (say 5K lines of code of less).

So there you have a really rough way of estimating cost based on Lines of Code and number of Software Developers involved.

The above is a very small example of the data collected by VDC Research so consider signing up if you want to see all of it.

Software Lines of Code

Software Lines of Code, or LOC, is only one measure of a project. There is much more to consider. We had a recent project where we were asked to fix 50K lines of code for a product that was proving unreliable. So I ran RSM over the top of it to get the average Cylcomatic Complexity and got 6.2! Those who know what that means probably have no idea how you could write code that hard to debug. And no, it wasn’t lots of switch statements. So is the correct answer $250K at $5 per line of code for a complete rewrite?

The answer is a resounding “NO”!

And the reason is because we redesigned the control flow and changed the UI to a table driven design and reduced a spaghetti mess of 50K lines of code down to 10K lines of cleanly designed code. Which is a budget more like $50K. In this case, it was much more cost effective to redesign from scratch than to try and rescue it. We also fixed the hardware design as that was in part responsible for the unreliability.

So the other answer is that good requirements analysis and good design will reduce Software Cost.

Successful Endeavours specialise in Electronics Design and Embedded Software Development, focusing on products that are intended to be Made In Australia. Ray Keefe has developed market leading electronics products in Australia for more than 30 years. This post is Copyright © 2017 Successful Endeavours Pty Ltd.

Programming languages

It is 2016 and we are a long way from the 1970s. So of course the world has moved on. Today’s programming has advanced significantly and we have super low powered systems of extraordinary capacity and easy to program securely. Surely!

Or should that be surely?

So here are the IEEE top programming languages for 2016.

Top Programming Languages 2016

Top Programming Languages 2016

This isn’t the first time we have done this. If you go back to Top Programming Languages 2015 you will see that C was second and Java was first. This year, C is first. A 1970s language is back to being first for all programming in 2016. Why?

It is called the Internet of Things or IoT.

So is this a step backward?

In 2016 devices shipped in the product category known as the Internet of Things exceeded all other mainstream electronics device categories. Take all mobile phones, tablet computers and iPads, desktop computers, netbooks, laptops, servers and general computing devices combined, and this is less than the value of products shipped in the category of the Internet of Things.

And this is what is driving the use of the C programming Language. For these small, low powered, low cost, essential to our future devices, have to be programmed in something that lets you get close to the hardware so you can manage it, and also operate in a high level language. My hat goes off to Dennis Richie and KenTompson who developed this language in the early 1970s and gifted it to us all. Their vision has carried an entire civilisation forward.

Ken Thompson and Dennis Ritchie

Ken Thompson and Dennis Ritchie

So more than 40 years later, I am still very grateful for their foresight, vision and competence in creating the most used computing language on earth today.

Successful Endeavours specialise in Electronics Design and Embedded Software Development, focusing on products that are intended to be Made In Australia. Ray Keefe has developed market leading electronics products in Australia for more than 30 years. This post is Copyright © 2016 Successful Endeavours Pty Ltd.

Top Programming Languages

We previously look at the Top Programming Languages for 2014 and IEEE Spectrum have released the same information for 2015.

Top 10 Programming Languages 2015

Top 10 Programming Languages 2015

One interesting thing is that use of C, the oldest of the top 5 languages, is increasing.

You can read the full article at The 2015 top ten programming languages including how they assessed which languages are used. They also have a App you can purchase if you want finer detail and to always be up to date.

Successful Endeavours specialise in Electronics Design and Embedded Software Development. Ray Keefe has developed market leading electronics products in Australia for nearly 30 years. This post is Copyright © 2015 Successful Endeavours Pty Ltd.

Product Development

As a process, Product Development can be handled a number of different ways. And if your product only requires input from a single technical discipline which you are very experienced in, then you can usually predict everything you need to do and just make sure it all happens the right way.

But if the product is complex, involves many disciplines, and has unknowns about the technical direction to take, then it can sometimes resemble a roller coaster ride more than it does a straight forward journey. And there can be unexpected bumps along the way.

Our most recent employee brought this video to my attention and I thought it covered this topic really well. We used it for an in house lunch and learn session so I recommend you check it out to. It isn’t short so you might want to set aside a time you can sit back and enjoy it.

The presented is Andrew “Bunnie” Huang and the conference he is presenting at is linux.conf.au 2013. 

Quite a list of things you can run into just getting a fully package embedded computing device ready for market. The HDMI Man In The Middle exploit was my favourite part.

Successful Endeavours specialise in Electronics Design and Embedded Software Development. Ray Keefe has developed market leading electronics products in Australia for nearly 30 years. This post is Copyright © 2015 Successful Endeavours Pty Ltd.

Top Programming Languages

A recent survey of the most used Software Languages, also known as the Top Programming Languages,  has revealed what most would have guessed as to the most popular Software Development Languages in use. The survey was published on IEEE Spectrum and usefully allows you to look at the statistics for 4 types of software development in any combination you want to. The categories used for the breakdown of the statistics are:

  • Web
  • Mobile
  • Enterprise
  • Embedded

Web Development Languages

The most popular Web Development Language was Java followed by Python, C# and PHP. I was surprised that Python ranked so highly. I understand the Google use it but hadn’t realised it had become so prevalent. So I learnt something new from that part. This is one of the advantages of being part of the professional body like IEEE. The world we live keeps changing and expanding and this is one way to stay up with those changes.

Mobile Software Development

For Mobile Software Development the winner was again Java with C, C++ and C# coming next. So here we are still very C oriented even on the most modern platform around. And of course Java is also very C like in its structure having been developed to address some system level issues such as memory management and garbage collection that C requires you to handle manually.

Enterprise Software Development

Enterprise Software Development sees Java again a the winner with C, C++, Python and C# coming next. Again Python is higher than I expected but the rest makes sense. Given that this area represents one of the core infrastructure requirements of modern scale-able companies it is interesting to see how narrow we still are with the Software Languages we use.

Embedded Software Development

And finally to the world we mostly deal with, Embedded Software Development. The clear winner here is C followed by C++ and Assembler. I took a snapshot of the complete rankings. Embedded C remains the primary language for software development in embedded systems.

Top Programming Languages - an IEEE published survey

Top Programming Languages

You can see the full survey results at Top Programming Languages

The overall winner was Java when you take all 4 categories into account and that isn’t hard to accept given Java was the Top Programming Language in 3 of the 4 categories. So if you are thinking about what Software Languages you should learn, it depends on where you want to work and what you want to work on. For Embedded Software Development, it is still C and C++. For everything else, Java is core with C, C++, C# and Python all playing a part.

Successful Endeavours specialise in Electronics Design and Embedded Software Development. Ray Keefe has developed market leading electronics products in Australia for nearly 30 years. This post is Copyright © 2014 Successful Endeavours Pty Ltd

iAwards

The iAwards are an annual celebration of Innovation in driving economic growth in Victoria. This year we were pleased to see one of our clients, Rectifier Technologies Pacific, nominate for an award. They were encouraged to do so and I was pleased that they followed through on that encouragement.

iAwards

iAwards

iAwards Winners 2014

The iAwards 2014 Winners for Victoria have been announced and Rectifier Technologies Pacific are winners in 2 categories:

  • Research and Development
  • Sustainability

This was for their RT15 240V 100A HRE Rectifier. This is a high power level and high efficiency AC to DC Power Converter.

Rectifier Technologies Pacific

Rectifier Technologies Pacific

One of the reasons we are familiar with the product is that we did some of the core Embedded Software Development and know just how good the hardware design in the power stage is. You can get more information from Rectifier Technologies Pacific AC/DC Product Range.

So congratulations again to Rectifier Technologies Pacific and to all the rest of the winners at this year’s iAwards.

Successful Endeavours specialise in Electronics Design and Embedded Software Development. Ray Keefe has developed market leading electronics products in Australia for nearly 30 years. This post is Copyright © 2014 Successful Endeavours Pty Ltd

Technology Selection

Before we look at how to choose a Technology, what does Technology mean?

In very general terms, Technology is understanding how stuff works and how to get it to do what you want.

Technology Selection

Technology Selection

There is lots of different stuff available. In the case of Electronics Design this stuff is the type of Electronics you will use and how you will make use of it. The most important choice to make is to determine:

  • What functions will I implement using Electronic Hardware ?
  • What functions will I implement using Embedded Software ?
  • What functions will I implement using Remote Communications ?

In looking at the answers to these questions I also need to consider:

  • Cost to Design
  • Cost to Manufacture
  • Cost to support
  • Production Volume
  • Power Consumption
  • Performance
  • Time to market

In the process of Product Development it is often Technology Selection that can make the biggest difference.

Electronics Hardware

If there is no software involved, then this is the choice of which devices can be used to implement the design and how best to use them.

Electronics Hardware

Electronics Hardware

A recent example for us was the interface and power supply for a new GPS module for the Yarra Trams Passenger Information Systems. There was a problem with the existing GPS modules in scenarios where buildings either side caused the GPS module to lose position. And guess what you have a lot of in the central part of a city? That’s right, taller buildings. The Passenger Information Systems required an accurate GPS position to work correctly. So the GPS module had been selected including the use of dead reckoning to update the position based on the wheel rotation and the interface between this and the rest of the tram had to be designed including some level shifting to adjust voltage levels. We also manufactured the interfaces for them.

Yarra Trams VPIS

Yarra Trams VPIS

So that is an example of a project that required no Software.

But most of the time there will be Embedded Software involved. And there are several really good reasons for this:

  • Embedded Software costs less in manufacture – see Reducing Electronics Manufacturing Parts Cost
  • Embedded Software is extremely flexible
  • Embedded Software can test itself
  • Embedded Software improves field support, service and upgrade capability
  • The Electronics Hardware to run Embedded Software gets cheaper every year
  • Remote Communications is getting cheaper all the time

So today we spend 80% of our time writing Embedded Software in C and C++ to run on the Electronics Hardware we design through the PCB Prototype or even Production. This is known as an Embedded System.

For this typical project type we do as much in Software as we can.

Embedded Software

Embedded Software is the software that runs on the Electronics Hardware. Unless the product must be super Low Power Electronics, we will do everything in Software except for the power supply and physical interfaces to the outside world. But there are a few caveats:

  • signal filtering is usually more power effective in Analog Electronics than DSP
  • sleep and wake timing for high powered systems is often best done with external Electronics Hardware
  • you have to be able to select a Microcontroller that has the right combination of price, features and performance
Embedded Software

Embedded Software

Given the enormous range of devices available today you would think the last point was easily covered but a recent project we did ended up with only 1 possible choice in the whole world for the Microcontroller. Here is the requirements list:

  • Run from a button cell for at least 2 years
  • Has a beeper
  • Has an LED
  • Operated from -20C to +70C
  • After a period of dormancy, start flashing the LED and activating the beeper
  • Beeper frequency, on time, off time, number of cycles and gap time are configurable
  • LED on time, off time, number of cycles and gap time are configurable
  • Dormant period is configurable
  • Unit timing must be accurate to better than 1 hour per year
  • Unit price in 100K quantities must be less than US$1
  • Software must be protected from copying

The solution was an MSP430 based device from Texas Instruments with a 32KHz crystal. Actual cost ended up at US$0.71. And absolutely everything was done in Software.

Remote Communications

 With ubiquitous Internet enabled devices, knows as the Internet of Things or IoT, it is more cost effective than ever to add Remote Communications to products. This can have many benefits that reduce the cost of field and service support for a product and also makes possible features you could not have provided any other way.

Remote Communications GSM Modem Cinterion

Remote Communications GSM Modem

An example from a recent water metering project we undertook. This is a remote water dispensing system, also known as a Bulk Filling Station, that records who took water, how much water, when and where. The transaction is sent to a website via GSM modem and the Council can get the records to bill for the water without having to travel. It also means the tanker drivers don’t have to manually fill out log books and the Council don’t have to chase them for the data. Great savings there alone. But there were some extra benefits for us and the client that they hadn’t considered. These were:

  • Remote updates to the system application
  • Maintenance monitoring of batteries and valves
  • Regular check in to confirm the system was still operational

So if a new feature is needed, we can update the software and remotely distribute it the units in the field. Since these are currently spread over half of the east half of Australia that is an enormous saving. 

Internet of Things - IoT

Internet of Things – IoT

And we can also determine when the batteries need to be swapped out so that can be a preventative maintenance operation at a time of the Council’s choosing and not an emergency call out when a truck driver can’t get water. It is quite common for the first tanker to fill up before dawn when the solar charging has been off overnight and the temperature is at its minimum for the day. The worst timing from the batteries perspective so it just works better all round if we known for sure how the batteries are travelling by keeping track. It also means that if a solar panel is damaged the Council can see there is an issue before the system stops working.

And the regular check in allows the Council to know if a unit is still operational or not. A recent example from NSW was a fire fighting crew going to a water dispensing point to refill their tanker during a bushfire only to find it had failed sometime last winter and never been repaired. With Remote Communications you can avoid that and although it costs more to design,manufacture and operate (due to SIM costs) it can still reduce the overall cost of a system significantly.

So that is the general process. Once we have decided what we will do in Electronics Hardware, Embedded Software and how much Remote Communications to use we are ready to get into the Electronics Design in detail.

And of course, no post like this is complete without an input from Dilbert.

Technology Selection - Get It Right

Technology Selection – Get It Right

Successful Endeavours specialise in Electronics Design and Embedded Software Development. Ray Keefe has developed market leading electronics products in Australia for nearly 30 years. This post is Copyright © 2014 Successful Endeavours Pty Ltd.

Estimating Software Development

This is actually a huge topic so I am just going to touch on some high level points. What prompted me to start writing on this topic is a recent conversation with a client and a cartoon I came across at Gizmodo about what happens when your Boss Estimates Software.

How long to write the software?

How long to write the software?

The question of how long it will take to write a particular piece of software is dependent on several factors:

  • what is your direct experience in writing that sort of software?
  • is there existing code you can leverage? – Software Reuse
  • build versus build – can you buy a module?
  • what is the Operating System? – an RTOS will usually slow things down
  • is it a hard or easy problem?
  • is it well defined?
  • how will it be tested?
  • what quality standards does it have to comply with – eg. Medical Device Class C
  • how many people will be working on it?

I’m sure you get the idea.

And the coding is just part of the Software Development Process. That is the thing that gets forgotten more than anything else.

Software Development Process

In the Software Development Process, coding is preceded by:

  • user requirements analysis
  • product requirements analysis
  • technical analysis
  • solution selection
  • specification
  • test methodology
  • Software Design

Then there may be a Design Review.

Then we code (sometimes referred to as putting the bugs in).
Then we test and debug (getting the bugs out).

Then there may be a Code Review and Refactor followed by confirmation it still passes all the tests.

Then we complete the Software Documentation package and create a labelled revision so it can be properly released and tracked.

That is the small software team version of the process and for some projects some of those steps are trivial.

Larger companies have larger processes but can also do larger projects as a result.

Industry Metrics for Coding

Here are some really basic Coding Metrics.

High security, financial systems, mission critical code – as little as 10 lines of fully debugged and documented code per day averaged across the whole process.

Commercial and scientific software is usually created at a rate of between 100 and 1000 lines of code a day.

And better processes actually speed that up rather than slowing it down.

Estimating Software Development Time

So a recent conversation with a client was on the topic of redoing someone else’s code. They had been working with another Software Development company and had decided that the code needed to be done again. They had spent 2 years without getting to a fully working version. My first though was “commendable patience”. My second was merely “ouch”!

So we did some analysis. I was initially optimistic. We used a tool called RSM to do some code base analysis. We use quite a lot of science in our Software Development Process including Static Analysis, Code Quality Analysis and complexity measurement. What we got from the initial analysis was 50K lines of code with an average Cyclometric Complexity of 6.21. The normal rule of thumb is that anything above 5 should be redesigned. Not Good. Then we looked at some specific files that had really high complexity scores above 10. That was the clincher. No evidence of design, no consistency, lot’s of cut and paste and everything is global variables.

The good news, is that the real complexity of the required code will not require 50K lines of code when it is properly designed. The bad news is that our client was right. It did need to be done again from scratch. Some parts might be reusable but it was unlikely.

Assuming we can do it with 20K lines of code, this will take between 20 and 200 person days to produce. In our case closer to 20 person days because the thing that makes the biggest difference to software delivery on time is your Software Development Process.

So that is a really quick look at a really big topic.

Successful Endeavours specialise in Electronics Design and Embedded Software Development. Ray Keefe has developed market leading electronics products in Australia for nearly 30 years. This post is Copyright © 2014 Successful Endeavours Pty Ltd.

Embedded Development

This is the second in a series of interviews between Ray Keefe of Successful Endeavours and Lance Harvie of People 4 Embedded. Lance is a LinkedIn connection of mine has wanted to do a series of interviews with Embedded Developers to go over issues, tips and career advice from those of us who have been in the industry for a while. We touch on topics such as:

  • Operating Systems
  • Software Testing
  • Design Methodologies
  • Hardware versus Software and trade-offs. 

 The first Interview is at Embedded Development.

Successful Endeavours specialise in Electronics Design and Embedded Software Development. Ray Keefe has developed market leading electronics products in Australia for nearly 30 years. This post is Copyright © 2013 Successful Endeavours Pty Ltd.

Next Page »