Flirting with sentimental analysis: my own approach and some case-scenario applications

Lately I have been interested in applying data analysis to information sources, particularly Twitter. Twitter has all the necessary features to provide an effective real time analysis: the API they provide allows us to access all the required features for analysis, and the volume of information is just huge. I strongly believe that Twitter has already changed the way to perform intelligent analytics, since it just contains millions of thematic tweets that can be accessed with no limitation.

Since I began to write my own intelligent agent for SCAI (a tournament to develop an AI agent able to play Starcraft) I really got very interested in modeling human intelligence. So of course there is a lot to do in this field, and doesn’t make any sense to list those well-known challenges still open. One of the fields that always attracted me was the Sentimental Analysis. How interesting is the idea of extracting sentimental information of texts? Furthermore, this field deeps into many interesting areas of the Artificial Intelligence.

A few months ago I began to develop my own Sentimental Analyzer. It wasn’t easy at the beginning: there are so many different approaches that the task seemed overwhelming. Since the project aimed to have a little more impact that traditional sentimental analyzers, I used a different approach: most of the ideas use keywords approach. They are pretty effective, but they do not ensure recall and it is more difficult to train the analyzer (i.e., train it automatically by using Twitter as a learning source). So I decided to use it using classifiers from different machine learning algorithms. The analyzer has been trained only for English, although it learns every day and I’m considering to expand it to some other languages (fundamentally Spanish and German)

And the experiment is working, and pretty accurate I would like to say. I have developed two applications to show them as testbeds for the analyzer:

  • The Happinness Observer, which determines the happiness for a certain concept
  • City Mood. This application determines the position of the user and throw information of the mood state of the city, based on the tweets from the people referencing it.

Feel free to download the applications and let me know what you think.

Last but not least, I have decided to give free access to the API for those who are interested. The idea is quite simple: you can call to this URL:

In the parameter text you should send the text (HTTP codified) you want to analyze. You will need to register for a key to use the application (and add it as a parameter on the same way: &key=providedKey). Drop me a line if you need a key and I will provide you one.

The application return a value corresponding the the polarity of the text: 0 means negative, 2 means neutral and 4 means positive. As a further enhancement of the algorithm I’m planning to extend the polarity of the analysis to support multi-values (i.e., not just providing discret values but a full continuous range of them, from 0 to 100 for example).

Enrique López-Mañas

How intelligent should the AI be

Reading and thinking the last days about how to implement an intelligent system to play Starcraft, I had time to think about the implications of considering a system “intelligent”.

Nowadays, we can develop systems that are able to defeat human intelligence in certain genres. Some board games like chess, or most of the shooters are composed by a limited set of rules, that can be easily modelated and represented with different combination of techniques (an expert system, considering most of the rules for almost all the situations is a typical choice). As my colleague Bruno points out (Spanish link), we even have to limit the intelligence of those systems by applying “stupidifying” techniques. In one of the examples Bruno provides, we only allow the AI to attack the human player after spotting him; leading this into very weird situations like the one exposed in the following video

But this does not apply in strategy games. Generally, a human player has much more capacity for analysis and improvisation in complex systems like strategy games, so the only way to empower the AI is to make it able to cheat (whether in the form of direct benefits, or having access to information that the human player can’t access in the same situation). The approach of experts systems, which has been proven to work well in most of the scenarios, can’t be applied here: implementing all the possible rules for one single (and simple) situation is an overwhelming task. Cheating AI are a well-known aspect of Sid Meier’s Civilization series; in those games, the player must build his empire from scratch, while the computer’s empire receives additional units at no cost and is freed from most resource restrictions.

How to solve this? Well, one idea could be to apply learning techniques. In the last AIIDE competition, the Berkeley team designed a special training field, aptly named Valhalla. Instead of manually adjusting the parameters of the AI, they let the AI fight on its own for a high number of iterations, letting it analyzing its own results and adjusting the parameter on its most convenient way. The result can be seen in the following video: they trained a group of mutalisks to massacre their natural counter unit, something unlikely if a human were controlling the units.

But we are still far away from reaching an AI that can defeat humans in complex scenarios. Although its limitations, Moore’s law is helping us, but we are not only facing a problem of computing capabilities. We also need to find different ways to model complex scenarios that the human mind is good at analyzing, but the computer is still behind us. And if we finally use cheating to defeat a human intelligence, we have to make it completely transparent: the user will not care as long as beating the AI is still challenging for him.