How to Effectively Test mobile applications To Optimize Energy Consumption

Why energy is the key factor for mobility

Smartphones and tablets allow people to carry around more computational power in their hands than most had on their desktops just a few years ago. However, the usability of these devices is strongly defined by the energy consumption of mobile applications, and user reviews of applications reveal many customer complaints related to energy usage

A modern mobile application brings increasingly complex functionality into mobile devices that are rapidly overtaking personal computer as a primary computing. Because of the battery/energy constraints of mobile devices, energy efficiency fitting an energy budget and maximizing the utility of applications under given battery constraints has become an important software design and maintenance consideration. Traditionally concerned with performance and memory usage optimization, perfective maintenance now has to address the challenges of optimizing energy consumption, with existing perfective maintenance techniques being mostly inapplicable

Key factors contribute mobile handset energy consumption

The main factor of mobile application energy consumption is the mobile network itself. Not like ordinary desktop applications, Mobile handsets use wireless state machine to manage network calls. Behaviour of the wireless state machine is totally depending on the network profile. For example the way that 3G network data calls utilize energy is not the way that 4G/LTE data calls utilize energy.

How 3G wireless state machine consumes energy

In 3G wireless state machine, it has three states.  They are IDEL State, CELL-DCH state (High Power Mode) and CELL FACH (Low Power Mode). CELL-DCH state requires 800mW and CELL FACH require 460mW per second.

When mobile application wants to initiate 3G data call, first it has to open a 3G network connection. According to 3G profile, opening connection is taking 2 seconds [1]. When opening a 3G connection our handset moves from IDEL state to CELL-DECH state [2]. Then connection will be established and request will be send to backend and handset is waiting for the response [A]. Mobile device remains in DCH for 5 seconds for the response. Sometimes response [B] takes more than 5 second and this idle time of 5 seconds is called the DCH tail. If no more data arrives, the wireless state machine switches to a CELL_FACH state [3]. The FACH state can handle signaling packets, but if more data comes in [C], the radio switches back to the DCH state. If no packets arrive after 12 seconds in the FACH state, the radio returns to the IDLE state and turns off.

With using above factors we can calculate actual energy requirement for each mobile transaction by adding actual time spent for DCH, FACH, IDELà DCH and FACHàDCH. Following is a sample of such calculation.

3G Energy Consumption
RCC State	Energy Consumption (in Joules)
DCH Active	xxxx
FACH Active	xxxx
IDELà DCH	xxxx
FACHà DCH	xxxx
Total Energy	xxxx

  

How LTE wireless state machine consumes energy

LTE wireless state machine has only two states. They are RCC CONNECTED state and RCC IDEL state.  RCC CONNECTED state is the high power mode and it require 1000 to 3500mW and RCC IDEL requires less than 15mW. There are two sub sates in LTE state machine with in RCC CONNECTED state. They are called Short DRX and Long DRX Both of these modes has a parent mode call Continuous Transmission state.

The RCC CONNECTED state that the wireless state machine uses when it is actively transferring data. This high power and high bandwidth state is what gives LTE its speed.

While in RCC CONNECTED state the wireless state machine creates the connection [1}. Then it initiates the request and waits for its response [A].  When data transmission is completed, the radio moves to the Short DRX state, but is still using high power while it waits for more data. If more data arrives, the wireless state machine returns to the Continuous Transmission state, and if not, it moves to the Long DRX state. While in the Long DRX state, the wireless state machine prepares to switch to the IDLE state, but is still using high power and waiting for data. If more data arrives, the wireless state machine returns to the Continuous Transmission state, otherwise it goes to low power IDLE state and turns off.

Using above facts we can calculate LTE energy consumption and following is the sample of LTE energy consumption calculation.

LTE energy consumption
RCC State	Energy Consumption (in Joules)
IDELà Continues reception	xxxx
Continues reception	xxxx
Continues reception Trail	xxxx
Short DRX	xxxx
Long DRX	xxxx
IDEL	xxxx
Total Energy	xxxx

Deferent between 3G and LTE State machines energy consumption

In the 3G state machine, the power drain in the DCH state is basically constant for all files. In LTE, the power drain in the Continuous Reception state varies depending on the throughput. Small files keep the power low, but as the throughput increases, the power gets even higher. Therefor LTE state machine drain slightly more power than 3G, because the tail states (Short DRX and Long DRX) stay at the higher base power, while much of the 3G tail is in the FACH state which uses half power.

Modem end uses expectation of a mobile application response id less than 3 second. This will make usage of LTE more. But LTE requires more energy than 3G. This requires QA professions to effectively test there AUT on 3G and LTE and calculate energy consumption effectively and find ways of optimizing.

 Best Practice Recommendation

Test for insufficiency due to scattered data transfers

In many mobile applications, files/data are requested with large gaps. Because of this actual data transfer through state machine will be through scatted way. Please note following diagram.

QA should be able determine the gap between scatted data transfer so development teams can make sure scatted data are tightly grouped. This will enable less time consumption in 3G or LTE sate machine.

Test for insufficiencies from delaying closing connections

In case open connections are not closed properly and effectively, 3G and LTE state machines are assuming some more data are on the way and they are in receiving modes.

Please refer to above diagram. Actual data burst was completed in 315^th second and developer actually closing the connection in 320^th second. Because of this, state machines waits unnecessarily in receiving mode for 5 secounds and it will utilize energy depending on the type of the connection (3G or LTE). Assume that end user travels to this page/screen for 3 times during his usage actual energy waste is three times as your calculation.

Test for content insufficiencies

Mobile screens are comes with different sizes and resolutions. Images use for each screen size and resolutions are needs to optimize. Otherwise this will cause unnecessary data transfer and require more energy.

Image on the left side of above takes 43 seconds to download. Next image looks identical but takes 3.5 second to download. This significant gain achieved simply by reducing the image resolution.  3G or LTE sate machines do not require additional time to download the second image and this will result low energy consumption.

Test For periodic transfers

Periodic transfers are specific packets that are sent repeatedly by a server over an infinite time horizon. They typically occur at regular intervals.

Periodic transfers ([P1] [P2] …. [P6]) trying keep the connection for long time but actually state machine after its time out disable the connection and again create a new connection ([1] [2] [3] and [4]). This will require additional 2secound time to create the connection and it use 2 into 800mW of energy. Therefore Periodic transfers needs to be optimized to minimize no of connections and QA needs to monitor them effectively to determine following

1. Best timing for Periodic transfer so that application uses as long a period as possible between transfers.
2. Feasibility of combine Periodic transfer with other data.
3. Feasibility of bundle as much data as possible into a single message.

Test for duplicate content

If mobile applications calling for same content form the back-end over and over then it should be cached. By doing this we can stop unnecessary data download and save energy.

Test for uncompressed file transfer

If mobile applications use file transfers then it has to be compressed. By doing this we can stop unnecessary bandwidth usage and save energy.

Most important factor of mobile handset optimization - Memory Test

 

Why Memory Test?

Mobile handsets have very limited memory compared to modern computers, and mobile operating systems have a default behavior to terminate applications that are using excessive memory and causing a poor user experience. Because of this memory testing is exceptionally important for mobile applications to ensure that each mobile application maintains optimized memory usage throughout the application life cycle.

However mobile application memory test is not that simple. This is mostly because each mobile operating system and its memory management mechanisms are not standard and unique. This is the main reason why some mobile applications smoothly run on Apple IOS devices but crashed in Android devices.

Testing mobile application memory is not simple like mobile functional test. Most of the people depend on testing tools for this. However you should understand how your mobile OS manage memory and adopt this knowledge into your test strategy is the success factor of your test. This white paper is discussing simple test strategy for Android and IOS mobile memory test.

Issues and Risks

Mobile OS Behavior and Memory Leaks

Memory is something totally belongs to the Kernel of the system. Therefor memory management is Kernels responsibility. As mobile developers we have limited options to control memory management. 

The biggest memory killer is memory leaks, Observations or defects you find during your memory test are mostly said memory leaks.  Most of the testers use some test tools and monitor mobile Ram and if Ram usage is 100% for a transaction they raise the red flag. This is totally wrong assumption in terms of memory leaks.

Memory leak are something that your mobile applications reluctant to clear its own memory objects witch he creates. Therefore memory leaks are totally depending on OS kernels ability to clear application HEAP.

Android kernel uses garbage collector (GC) for memory clean. This is mostly because Android is Java and Java use lots of objects in the memory in order to execute your application. This is what developers refer as classes and there instances. GC needs a lot of head-room to run. Depending on the type of GC, you really need 2 times to 8 times more memory than you are actually using to get good performance. When memory is constrained the system slows down dramatically. This is why Android applications usually use more memory then IOS Applications.

However Objective C based IOS dose note use same approach. IOS was designed from the outset to be memory efficient and avoid "Garbage Collection" of this sort. Hence, the iPhone can run faster on lesser memory and is able to deliver similar battery life to that of many Android phones boasting vastly larger batteries.

More Memory Mean More Energy

Android tends to keeps all your opened applications alive in within RAM, if there is enough free RAM to do it. This increase energy consumption because your applications are only initialized on the first run, and then kept alive while you use your phone. These applications are however, not using any energy while kept on your RAM, as the processes are generally not running unless background processes exist.

For Android, more RAM means more applications can be kept alive, thus faster switching between applications for the user. Because of this Android tends to waste empty RAM. However when Android need RAM and he don't have it, Android will kill automatically the less used processes until he have enough RAM to run active applications.

IOS does the same thing, all recently used applications remain in memory in a suspended state, so it comes down to how Android and iOS handle these suspended apps.

Usually the limitations enforce by IOS restrict developers and what they can do causes them not to do much harm, while the freedom Android gives them often causes memory leaks or resource hogging background processes. Android has made some attempts to control this, such as killing apps that use too much CPU, but still, these problems can be noticed by the user. This coupled with the fact that every phone manufacturer adds its own apps when they ship your Android device means every manufacturer can potentially add an app that will hog the memory. With iOS you only have to worry about Apple, and they usually undergo throw testing.

Development Best Practices Vs Memory Leaks

Android uses Java which promotes a lot of good practices which optimize the Java code specifically for Android devices, reducing the size of the app, the memory usage, and the impact of Garbage Collection. However, in order to do this, Android declares a lot of Java's good practices as bad practices, something not all developers are aware of, which means they think they are writing good code, when actually, it's quite the opposite. This is why garbage collection has such an impact.

IOS does not have this issue since there development environment xCode has enforced their best practices and the application runs on controlled environment witch generally tested by Apple. However modern hybrid development frameworks such as Titanium, PhomeGap also generate IOS and Android applications. These applications use underlying Framework libraries. This needs to consider for your test strategy.

 Test Approach

Android and IOS ships with debugging tool called Android DDMS and IOS xCode Instruments. Both these tools are capable of monitoring your application memory usage. Therefore we are not going to discuss how to read said data form the tool but we will discuss how to use them and find a leak.

With the information we discuss in “Section 2” of this whitepaper, we should know following information about our AUT in order to plan our memory test.

1.       Is our AUT use bitmaps and where

2.       Code abstractions and witch workflow actually execute it

3.       Use of external libraries and execution points

4.       Use of native hardware calls and execution points

5.       Performance best practices followed by the development team

6.       Background process if any and there execution and termination points

Then you need to identify your test scenarios which cover above points. These are your test cases for the memory test. Then you should understand the application lifecycle. Application life cycle is defers form mobile OS to OS. However for the simplicity of the test we can assume following 5 stages

1.       Not running - the app has been terminated or has not been launched.

2.       Inactive - the app is in the foreground but not receiving

3.       Active - the normal state of "in use" for an app

4.       Background - the app is no longer on-screen but is still executing code

5.       Suspended - the app is still resident in memory but is not executing code

When press the home button, the app moves from Active to Background. Most apps usually then go from Background to Suspended in a matter of seconds. The first technical caveat is that Suspended apps remain in the device's memory. This is so they can resume more quickly when you go back to them. They're not using processor time and they're not sucking battery power.

 Test Execution

Then use above behavior and identify your test execution plan. You must cover at least above 5 items during our memory test. Then use either Android DDMS or xCode Instruments to measure following for each test execution cycle. You need to call GC manually before you take each reading.

1.       Heap Size in (MB)

2.       Allocated Heap (MB)

3.       Free Heap (MB)

4.       No of Objects

Your Test execution will be as below.

Time	Test Activity	Heap Size (MB)	Allocated Heap (MB)	Free Heap (MB)	Heap Used (%)	No of Objects
12.45	Log into Dash Board and stay for 5min	19.566	16.641	2.926	85.05%	55509
13.45	Push app into Foreground and wait 5min	19.566	16.636	2.931	85.02%	55207
13.50	Push app into background and wait 5min	19.566	16.633	2.933	85.01%	55192
13.55	Push app into Foreground and wait 5min	19.566	16.646	2.921	85.07%	55430
14.00	Push app into background and wait 5min	19.566	16.633	2.933	85.01%	55192
14.05	Push app into Foreground and wait 5min	19.566	16.648	2.919	85.08%	55475
14.10	Logout	19.566	16.639	2.928	85.04%	55277
14,15	send the sign in screen to BG and bring back to FG, Then stay in login screen	19.566	16.639	2.928	85.04%	55278
14.20	Stay in login screen	19.566	16.639	2.928	85.04%	55279
………	……………………………….	……	……	……	………	………
………	……………………………….	……	……	……	………	………
………	……………………………….	……	……	……	………	………
16.50	Logout	19.566	16.64	2.927	85.04%	55294
	Await 2 hrs. to clear heap

Analyzing Test Results

Identifying Memory Leak

Once you collect above data then you can plot below graph and check witch test scenario fail to clear its heap after GC.

 

Please refer to above charts. At the time of the application starts 7.023MB of Heap (A) was used and when sending the application background and doing some other task (Take photo) and calling the application back to foreground has increase the heap usage to 7.148MB. However after logout Heap usage was 7.058MB. Then I leave the handset for 2hours while calling GC for 15min intervals but suspected leak was not cleared. This way we can identify witch test scenario/application work flow is leaking memory

 

Analyze Heap Dump

Now we need to get a binary dump of the heap memory using either Android DDMS or xCode Instruments and analyze for witch objects or class actually cause this problem.

There are several open source heap analyzers in the internet and you can download either one and open your AUT heap dump. Then create suspected memory leak report and Object usage report to identify witch object stayed in the heap for longer period of time.