This weekend I spent some time making a simple mobile app for emotion recognition, using Microsoft Cognitive Services. There are several cool APIs there, for instance the Computer Vision API which can recognize content in an image, recognize celebrities, and extract text from images. I guess I just ended up with the Emotion API because it also might be useful for work.
The APIs are well documented, and there are client libraries and sample applications on GitHub, and the client libraries are available at Maven central repository for Java and NuGet for C#. And they are free of use for registered developers, within some limits (30000 transactions pr month for the Emotion API).
I first decided to try to make the app in Xamarin, since it’s now free with Visual Studio, but I didn’t get the the client library to work there, it seemed to be something wrong with serialization in mono, so the library might not be entirely cross-platform (or I messed up somewhere). So I switched to Android studio, and made the very simple demo app there instead.
The app allows you to take a photo, and the photo is then posted to Microsofts service for emotion recognition The service returns a bounding box for each recognized face, together with confidence across the eight feelings; happiness, sadness, surprise, anger, fear, contempt, disgust and neutral.
The initial testing showed that the old doll is neutral, but it also scores some at surprise. The clown was happy with very high confidence, 0.99997, but the clown scored similar on happiness without the nose. Further, the faces of teddy bears and guinea pigs are not recognized, and the same was occasionally true for my husband, too much beard perhaps?
Yesterday at work we discussed mobile app development on different platforms and technologies. Someone mentioned Sencha, which I hadn’t heard about before, so today I had to find out more 🙂
So here are some notes on what I did to get it to work on Ubuntu.
* I started with this list for getting started with PhoneGap
* When I wanted to install the ADT plugin in Eclipse I got an error saying “missing org.eclipse.wst.sse.core 0.0.0”. I then did the steps of post #7 here which did the magic.
* After all the installing I continued with the “Hello World”-example here, which also gives a good setup for following the Sencha tutorial. (One might name the project Contacts right away)
* I happily continued with the Sencha tutorial for a while, copy-pasting code and getting the folder structure as described. But after a while the need to test the app on a real device emerged.
* So to get my Samsung galaxy to communicate with the computer I first put the phone in developement mode in settings -> Programs -> Development, and then I did the story described in this forum post, and restarted eclipse, and then suddenly, the app ran on the phone rather than on the slow emulator.
* After that, I really wanted the app to get and display my phones contact list, but I was only getting an empty page, and an error in the log saying “Uncaught DataView requires tpl, store and itemSelector configurations to be defined.”, and of course, the order in which the js-files are included in the index.html matters! Hence, include Contact.js before ContactsList.js, and then, ta-daa, the app is listing my contact list! (The rest of the tutorial will be food for another day 🙂 )