Friday, December 19, 2014

Encounter with CTS..!!

So finally coming back to my blog which was long due.

For a software professional like me and that too a bachelor living with his crazy friends in Pune, time is always on the fly. 24 hours seems like nothing for a day and interestingly this feeling remains throughout the year. But writing my thoughts is my happiness dose to myself, be it on nature, emotions, social cause, patriotism, hot issues, any idea or any technology. I have been writing up my thoughts ever since I was in my 9th standard, I am still novice in penning down my thoughts but that does not stop me from transforming my thoughts into my words. After writing previously on love, life and moral duties, this time I am writing on a technical topic for which I got inspiration, motivation and also some persuasion from one of my colleague.

CTS, no I am not referring here to Cognizant Technology Solutions, CTS is a small test suite containing around 99K test cases (latest Android L) to be verified on an Android device to make sure it is compatible to be sold to public. CTS – Compatibility Test Suite. I first came across this term around 2 years back while I was working on a project in my company. At that time my company was trying to make a similar project on a small scale that was specific to some devices which we were working upon in the project. I did some findings about CTS on internet but didn’t take it seriously then. One and a half years later CTS again came into my life and this time I had to take it seriously.

I gained the knowledge of it from my very special friend cum colleague for whom CTS executions was like a mania when he started working on it. I can tell you with all my personal experiences while working on CTS that it is hell to handle the failure cases until you become smart enough with its handling. My colleague somehow managed to make its execution understandable to me. Then I started reading about CTS on internet and I understood its importance. Even though it’s not necessary for one to execute and run CTS each and every day but it’s important for a software professional working in mobile domain company especially on Android Architecture to know what exactly CTS is and what exactly CTS does.

Android OS from Google is used by many device manufacturers all over the world for running their handsets, tablets etc. There are thousands of device based on Android OS that are sold in the market. It would be impossible for Google to verify the compatibility of each and every handset (running on Android) from different companies.  Without a strong compatibility standard, devices can vary so greatly that developers must design different versions of their applications for different devices. So Google came up with CTS, a testing tool which is free, open source and available for download. It is designed to be used for continuous self-testing during the device development process. Android devices will differ in hardware and software capabilities, so the compatibility program also provides the tools needed for distribution systems such as Google Play to implement appropriate filtering. This means users see only the applications they can actually run. The Android compatibility program makes it easy for mobile device manufacturers to develop compatible Android devices.

How does the CTS work?

The CTS is an automated testing harness that includes two major software components:

  • The CTS test harness runs on your desktop machine (linux machine) and manages test execution. 
  • Individual test cases are executed on attached mobile devices or on an emulator. The test cases are written in Java as JUnit tests and packaged as Android .apk files to run on the actual device target.

CTS setup and execution:
  1. Download CTS zip and CTS media files zip (Download CTS zip as per the version of Android device you are using)
  2. Attach a device or an emulator to the machine. Unzip the CTS zip and CTS media zip.
  3. There will be 4 folders after unzipping CTS zip namely docs, repository, resource and tools. Unzip CTS media zip and you will find CTS media files to be used while execution.
  4. Create a new sh file or edit the file present in CTS media zip to push each and every resolution media files into the device. (basically adb push all the media formats file into the device sdcard)
    For eg: adb push bbb_short/1920x1080 /mnt/sdcard/test/bbb_short/1920x1080
  5. Ensure adb is in system path.
  6. Now from terminal, adb install –r android-cts/repository/testcases/CtsDeviceAdmin.apk
    After that go to device -> Settings -> Security -> Device Administrators
    enable the two android.deviceadmin.cts.CtsDeviceAdminReceiver*
    Note: Make sure the android.deviceadmin.cts.CtsDeviceAdminDeactivatedReceiver stays disabled in the same menu.
  7. If the device has a memory card slot, make sure the device has an SD card plugged in and the card is empty.
  8. Set up your device with English (United States) as the language (Settings > Language & input > Language).
  9. Turn on the Location setting if there is a GPS or Wi-Fi / Cellular network available.
  10. Connect to a functioning Wi-Fi network.
  11. Make sure no lock pattern is set on the device (Settings > Security > Screen Lock 'None').
  12. Check the "USB Debugging" development option (Settings > Developer options > USB debugging).
  13. Connect the host machine that will be used to test the device, and “Allow USB debugging” for the computer’s RSA key fingerprint.
  14. Check Settings > Developer options > Stay Awake.
  15. Check Settings > Developer options > Allow mock locations.
Using the CTS:

The real encounter with the “MESS” the so called CTS will start from here.

Launch the CTS console by running the cts-tradefed script from the folder where the CTS package has been unzipped, e.g.

$ ./android­-cts/tools/cts­tradefed

For Android 5.0 CTS, java 7 or above is required.

After running the above command, one will enter into CTS tradefed. You may start default test plan i.e CTS.xml present in android-cts/repository/plans/CTS.xml using the command

cts-tf> run cts  --plan CTS

This command will run all packages present in CTS.xml. Individual package, class or even a method can be run separately.

cts-tf> run cts  --package
cts-tf> run cts  --class
cts-tf> run cts  --class –method testDownloadManager

While the test starts do not touch the device and keep it connected via usb to the machine. When all the test cases complete executing a final report in xml format will be generated. Test results are placed in a file and logs are also captured.

/android­-cts/repository/results/<start time>/testResult.xml
/androi­d-cts/repository/logs/<start time>

One can re-run any testcase failed or passed anytime to verify again with different set of conditions.

So this was an initial introduction about CTS and its setup and usage. While working on it, one comes around much informative stuff about Android OS, SEAndroid, Android APIs and many other things. For any detailed information about CTS go through the following links:

I would urge whosoever is interested especially those who are working on handsets/tablets development to setup CTS and run CTS at least once. Some points that I came to know with my experience of CTS
What is Tradefed?
Java version specific to CTS version.
Enforcing mode Permissive mode based on SE Android. Denials, audit logs, etc.