Software quality through automated testing
Continuous integration and continuous testing in embedded development are important for ensuring software quality right from the start. Maximum reliability is essential, especially for embedded systems—such as those used in medical technology or automotive applications. How can we achieve high software quality during development rather than only discovering errors at the end?
The answer lies in a well-designed CI/CD (continuous integration/continuous delivery) pipeline and comprehensive automated testing, from the smallest unit test to hardware-in-the-loop testing.
In traditional embedded software development, new functions were often only merged and tested at a late stage – a risky approach, as errors only become apparent at the end. With continuous integration, we are taking a much more efficient approach. Every time our developers write new code and import it into the repository, a process is automatically started that compiles the software and performs a whole range of tests. This automated process runs in a CI environment that we have specifically adapted to the requirements of embedded systems.
For embedded projects, CI is no trivial task, as firmware often has to be built for specific microcontrollers or processors and, if necessary, tested on real hardware. We have created an infrastructure that enables continuous integration even in embedded environments. Specifically, this means that automated tests are triggered at both the software and hardware levels as soon as an update is available. Build servers and pipeline tools translate the source code for the target system, load the new firmware onto test devices or simulators, and execute defined test cases – fully automatically and at any time. This gives us early feedback on the quality of each code change. Problems can be identified and corrected immediately, long before they become costly or critical later in the project.
From unit testing to hardware-in-the-loop
A CI pipeline is only as good as the tests it performs. That’s why we rely on a multi-stage testing concept that covers all levels of the software:
- Unit tests: We put the smallest units of code—individual functions or modules—through their paces with unit tests. These tests run in isolated environments (often on a PC) and ensure that each code component works correctly on its own. Thanks to modern test frameworks, we can run hundreds of unit tests in seconds, which is ideal for integration into our CI process. Any errors are detected immediately, while the developer is still working on the corresponding function.
- Integration tests: Once the individual modules are working, we test how they interact with each other. At this stage, we simulate a subsystem, for example, or run the software on a development board to ensure that interfaces and communication between components are working smoothly. Automated integration tests help us to detect incompatibilities or malfunctions in the overall system at an early stage.
- Hardware-in-the-loop tests: Here, we connect software and hardware to form a closed test loop. Our development and test environment includes special test benches and electronic simulators that replicate the real device or its environment. The current firmware is loaded onto the real target hardware or a hardware simulator and runs there under near-real conditions. Sensors and actuators can be emulated by simulators or physical test setups. This allows us to check whether the software works flawlessly in conjunction with the hardware – even before the first prototype is tested in the field. HIL tests detect, for example, timing problems, memory bottlenecks, or interface errors that would not be visible in pure software tests.
- Functional tests and stress tests: Beyond the stages mentioned above, we test the software in realistic scenarios. Functional tests ensure that all requirements are met from the user’s point of view – the device does exactly what it is supposed to do. In addition, we subject the system to stress tests in which we simulate extreme conditions: high load, rapid sequence of commands, temperature or voltage limits (as far as this can be reproduced in the laboratory). The aim is to find out whether the embedded software remains stable and reliable even under exceptional circumstances. Such automated endurance tests often run overnight or over the weekend in our test environment, so that we have comprehensive results by the next working day.
As soon as new code is available, the appropriate tests are automatically triggered. The result of each run is immediately visible to the development team. If a test fails, we know exactly which last set of changes is responsible—and can intervene directly. Continuous testing prevents errors from going unnoticed for long periods of time or multiplying in further development steps.
Why do we go to all this trouble?
In industries such as medical technology, automotive, and industrial automation, human lives and enormous costs depend on software quality. A single software error in a safety-critical device—such as a surgical instrument or a driver assistance system—can have serious consequences. Therefore, prevention is better than cure.
By integrating CI/CD with comprehensive testing, we achieve consistently high software quality throughout the entire development process. Errors are not only discovered at the end during quality assurance, but continuously, as soon as they occur. This has several advantages:
- Problems are identified and resolved at the earliest possible stage. This drastically reduces the cost of troubleshooting—a bug that is fixed at an early stage causes much less effort than a bug that is noticed shortly before product launch. For our customers, this means more reliable planning and fewer surprises shortly before market launch.
- In safety-regulated industries, strict standards must be adhered to (e.g., ISO 26262 in the automotive sector or IEC 62304 in medical technology). Our continuous testing and automatic code checks ensure that we meet these standards at all times. For example, we also run static code analyses in the pipeline that check the source code for compliance with guidelines such as MISRA-C/C++ or other safety coding standards. This ensures that the code is not only functionally correct, but also secure and maintainable in accordance with applicable regulations.
- By combining unit tests, HIL tests, and stress tests, we achieve very high coverage of possible error scenarios. The end product behaves predictably and stably because it has already been through virtually every critical situation in testing. For a medical device, this means greater protection for patients and users because the software is robust. For an industrial device, it means less downtime and thus greater satisfaction among end customers. This reliability does not develop by chance, but is the result of our systematic testing approach.
- Although extensive testing takes time, our approach shortens the overall development time in the medium term. Thanks to automation, tests run parallel to development without slowing down the team. On the contrary, developers receive immediate feedback on whether a change works or not. Time-consuming troubleshooting shortly before project completion is largely eliminated. As a result, projects reach their goal faster and with greater certainty. This is an important factor, especially for decision-makers in companies: time-to-market improves without compromising quality.
By firmly integrating automated testing at all levels into the development process, we create transparency and trust in the software – from the first prototype to the finished product. You are always aware of the quality status of the software and can be sure that we proactively ensure compliance with all requirements.
For you, our approach means above all a reduction in risk. When you commission B&W Engineering to develop your embedded systems, you can be confident that quality is not a matter of chance. Through continuous integration and comprehensive testing, we ensure that your product meets the highest standards – technically, qualitatively, and in terms of compliance with standards – even before it is launched on the market.
