During embedded software development, the ratio of time spent testing to time spent coding is usually around 3:1 (it may actually be more). This ratio will continue to decrease as engineers become more advanced in programming and testing, but regardless, software testing is a critical part of embedded software development.

Years ago, an engineer, in his quest to have a deeper understanding of embedded, asked the question, "How can I know and understand what my system is really doing?" . The most common questions asked by embedded developers of the same generation revolved around "how do I make my program run faster" and "what compiler is the best", which is an unusual but mature question. Today, let's learn 10 embedded development testing secrets that are widely spread in the industry.

01

Know how to use the tools 

Embedded systems usually have high reliability requirements, once a security problem may lead to catastrophic consequences, even if it has nothing to do with security will bring serious economic losses, embedded systems and software have strict testing, confirmation and verification requirements. As more and more fields of embedded devices begin to be controlled by software and microprocessors, fast and effective testing of increasingly complex embedded software becomes more and more important.

A good mechanic needs good tools, and a good programmer should be able to skillfully use a variety of software tools. Different tools have different scope of use, function. The right tool allows the engineer to see what the system is doing, what resources it is using, and what exactly it is dealing with in the outside world. Engineers should not be afraid to join the test tool or test module to the code requires skills or may introduce new errors, rely solely on continuous modification, recompile the code to eliminate bugs is not enough; should not be used to using printf and other simple means of testing and not to carry out new learning and exploration.

The following are some embedded commonly used testing tools.

Source-level debugger [Source-levelDebugger]: This kind of debugger generally provides single-step or multi-step debugging, breakpoint settings, memory detection, variable viewing and other functions, is the most basic debugging method for embedded debugging.

Simple and practical print display tool [printf]: printf and similar print display tool is estimated to be the most flexible and simple debugging tool. Printing various variables in the code execution process allows engineers to be informed of the code execution, but printf on the normal code execution interference is relatively large (generally printf will occupy the CPU for a longer period of time), you need to be careful with the use of, and it is best to set up the print switch to control the print.

ICE or JTAG debugger [In- circuitEmulator]: ICE is a device used to emulate the CPU core, can not interfere with the normal operation of the operator, real-time detection of the internal workings of the CPU, but also like desktop debugging software to provide complex conditional breakpoints, advanced real-time tracing, performance analysis, and port analysis, etc. ICE generally have A special kind of CPU, called bond-out CPU, is a kind of CPU that has been opened the package and can access the internal signals through a special connection, which can not be "seen" when the CPU is encapsulated. When used in conjunction with powerful debugging software on the workstation, ICE provides almost the most comprehensive debugging capabilities. However, ICE also has the disadvantage of being expensive and not working at full speed; similarly, not all CPUs can be used as external CPUs, and on the other hand, it is unlikely that these external CPUs will be replaced in time by newer CPUs. debugging support.

ROMMonitor: A small program that resides in the ROM of an embedded system and communicates with debugging software running on a workstation via a serial or network connection. This is the lowest-end technology, is relatively inexpensive, does not require any specialized hardware other than a communications port and a small amount of memory space, and provides functions such as downloading code, runtime control, breakpoints, single-stepping, and observing and modifying registers and memory. Since the ROM monitor is part of the operating software, if you want to check the status of the CPU and the application program, you must stop the application program and enter the ROM monitor again.

DataMonitor: Without stopping the CPU, DataMonitor not only displays the contents of the specified variables, but also collects and graphically displays the process of change of each variable.

Operating System Monitor: Operating system monitors can display events such as task switching, signal volume sending and receiving, and interrupts. These monitors are able to present the relationship and time connection between events, and can also provide diagnosis of problems such as signaling priority reversal, deadlock and interrupt delay.

Performance analysis tool [Profiler]: can be used to test where the CPU is consumed, to understand the system bottleneck, CPU usage and the need for optimization.

Memory Test Tool [MemoryTeseter]: It can be used to find out the problems of memory usage, such as memory leaks, memory fragmentation, memory crashes and other problems. If you find unpredictable or intermittent problems with your system, you should try using a memory testing tool.

OperationTracer【ExecutionTracer】: It can show what functions are executed by CPU, who is calling, what are the parameters, when to call, etc. It is mainly used to test the logic of the code, and it can find anomalies in a large number of events.

CoverageTester】：Mainly shows what code the CPU specifically executed, easy to understand the code branch is not executed in the region, help to improve the quality of the code and eliminate useless code.

GUI test tool [GUITester]: most embedded applications with a particular form of graphical user interface, part of the system's performance testing is based on the user input response time to carry out the GUI test tool can be used as a development environment to run the test cases of the scripting tool, its functions include the operation of the record and playback, capture the screen display for subsequent analysis and comparison, setup and management of the test process (Rational Corporation). GUI test tools can be used as scripting tools to run test cases in a development environment, with functions such as recording and playing back operations, capturing screen displays for subsequent analysis and comparison, and setting up and managing the test process (Rational's robot and Mercury's Loadrunner tools are prominent examples). Embedded devices without a GUI can be plugged in to run GUI test scripts, which requires changes to the code under test, but saves time in functional and regression testing.

Embedded semi-physical simulation test development environment [ETest]: is specifically for embedded systems test development tools, the system provides a graphical test case development environment, automatic generation of test scripts; test results data can be online monitoring, while generating test results information, and automatically generate test reports to meet the requirements; ETest for the open platform, providing C/C++, Python, Lua and J. ETest is an open platform, providing C/C++, Python, Lua and J. ETest is an open platform for embedded systems. ETest is an open platform, providing APIs such as C/C++, Python, Lua, JS, etc. The graphical monitoring software interface can be customized according to the user's needs; it supports various deployment schemes of domestic CPU+domestic operating system. Based on ETest, engineers can quickly build measurement and control systems for embedded devices, realizing real-time, dynamic, closed-loop, non-intrusive automated testing of the parts under test.

02

Early Detection of Memory Problems

Memory problems are hazardous and not easy to troubleshoot. There are three main types: memory leaks, memory fragmentation and memory crashes. To deal with memory problems, we must have a clear attitude of early detection and early "treatment".

Memory Leak

Memory leaks are the most common memory problem in software design, meaning that system memory is gradually depleted due to constantly allocated memory that cannot be freed in time. Memory leaks can be encountered even by careful programming veterans, as they are generally hidden and difficult to detect through code reading, and may even occur in libraries - either because the libraries are buggy, or because engineers have not properly understood the interface documentation and have misused it.

Most memory leaks, while undetectable, manifest themselves as random failures and are often assumed to be hardware problems. If users require high system stability, such problems can lead to loss of customer confidence in the product and project failure. Considering the great danger of memory leaks, there are numerous solution tools available to detect memory leaks by finding unreferenced or reused code blocks, garbage memory collection, library tracing, and other techniques, and although each tool has its pros and cons, you should take precautions and test for memory leaks as much as possible.

Memory fragmentation

Memory fragmentation has a deeper insidious nature than memory leaks. As memory continues to be allocated and freed, large chunks of memory are constantly being broken down into smaller chunks, resulting in fragmentation, and subsequent requests for larger chunks of memory may fail. A large enough chunk of system memory may or may not be able to hold out for a longer period of time, but ultimately there is no escape from allocation failure. Memory fragmentation occurs frequently in systems that use dynamic allocation.

The most effective approach to this problem today is to use tools that display system memory usage to find the culprits of memory fragmentation and make improvements. Many companies choose to avoid dynamic memory management problems by disabling malloc/free in embedded applications.

Memory Crashes

Memory crashes are the most serious result of memory usage and can be caused by out-of-bounds array accesses, pointer miscalculations, repeatedly freeing the same section of memory, and freeing non-dynamic memory. These problems are usually random, and are extremely difficult to troubleshoot in advance, and there are few tools available to do so.

In summary, memory management units must be used with care and strict adherence to the rules of their use.

03

Deeper Understanding of Code Optimization

The focus on embedded systems is usually on real-time and speed, two elements that directly affect code efficiency and require code optimization. Understanding how to optimize code is a must-have skill for every embedded software developer, and a prerequisite for optimizing code is to find out what really needs to be optimized and then treat it.

The profile mentioned above can record data such as CPU occupancy of each task, priority allocation, number of data copies, number of disk accesses, whether to call the network send and receive programs, test code has been closed, etc., but there are still shortcomings in analyzing the performance of real-time systems. On the one hand, profiles are often used when the system has a problem, i.e., after the CPU is exhausted, and the profile itself is CPU intensive, so it is very likely that it will not work. According to the Heisenberg effect, any means of testing will more or less change the system operation.

0

4

Don't look for a needle in a haystack

Finding a needle in a haystack is a vivid metaphor for debugging. When looking for bugs, it is important to check if you have ever failed to follow the coding design specification in order to take shortcuts, or failed to check the correctness of some assumptions or algorithms, or failed to mark code that may be problematic. Refer to the "Guide to High Quality C++/C Programming" or the "0x8 "scriptures" on C" for more information.

In order to expose and capture as much of the root cause of the problem as possible, you can design more comprehensive bug-tracking code: handle every function call failure as best you can, and check the validity of every parameter input and output as best you can, including pointers and whether you're calling a procedure too much or too little. Bug tracking gives an idea of the approximate location of the bug.

05

Emphasize and isolate problems

For large, module-independent projects, if the problem is intermittent, it is necessary to try to reproduce it and document the complete process so that it can be reused the next time the problem occurs.

Ensure that the problem can be solved by isolating it when it occurs: turn off code that may not be related to the problem with #ifdef, and minimize the system to the point where the problem can still be reproduced. If you still can't pinpoint the problem, consider opening your "toolbox": try using ICE or a data monitor to see what's going on with a suspicious variable; use a tracing tool to get a sense of what's going on with a function call (including parameter passing); and check for memory crashes and stack overflows.

06

Retreating to Advance

Hunters often leave markers on trees so that they don't get lost in the forest, and keeping track of past code changes can be helpful for debugging future problems. The code control system SCS or code control system SCS can be a good solution to the problem of backtracking, checkin the last version down and compare it with the current test version, which can be done by using the diff tool that comes with SCS/VCS/CVS or other more powerful comparison tools, such as BeyondCompare and ExamDiff. by comparing and analyzing all the changes to the code, you can get all the changes that may cause problems. By comparing and analyzing all the changed codes, you can get the analysis results of all the suspicious codes that may cause problems.

07

Determining test completeness

Coverage tests can be used to confirm the completeness of tests by confirming what code is actually executed by the CPU. The coverage tool has different test levels and the user can select a level according to their needs.

Even if the unit tests are comprehensive and there is no deadcode, the coverage tool can still point out some potential problems.

Take the following code as an example:

if(i>=0&& (almostAlwaysZero==0 || (last=i)))

If almostAlwaysZero is non-zero, then the last=i assignment statement is skipped and the goal cannot be completed.

<span