Cognitive Kinecting with the World

This post has been republished via RSS; it originally appeared at: Channel 9.

Shiny Ranjan &  Benedikt Lorch blog about their experience Kinecting in a recent hackathon...

Kinect & Cognitive with Kinecting the World – Guest blog from ICHack17 Microsoft Challenge Winners

During early February weekend, we set off as individuals for IC Hack 17 – a Department of Computing (Imperial College London) organised hackathon – with Microsoft being the gold sponsor. None of us knew what to expect. Through this article we want to share our experiences how the initial anxiety turned into a great surprise.

Animated by an exuberant opening ceremony, everyone quickly rushed past the sponsor’s free takeaways and immediately got started with the hack. Having arrived as individuals, the five of us found each other in the #teamfinding channel on Slack and soon enough, we formed a team.

We are:

· Joon Lee, 3rd year Philosophy and Economics, University College London

· Qingyue (Cheryl) Yan, 1st year Physics, Imperial College London

· James Knight, 2nd year Joint Mathematics and Computing, Imperial College London

· Shiny Ranjan, 1st year Computer Science, Queen Mary University of London

· Benedikt Lorch, 4th year Computer Science, Friedrich-Alexander-University Erlangen-Nuremberg

The idea

With each of us coming from diverse disciplines and backgrounds, we didn’t have difficulties in collecting ideas for a project. Even before everyone could vote for their favourite project on our list, we got intrigued with the idea of creating a Kinect-based language learning game that comes with training and game mode. We decided to follow through with this “Kinecting the World” idea.


Jump Start

Having been one of the last teams to assemble, we could not find a table with enough space and started working with laptops on our laps. However, the organisers were very collaborative, even under the stress of feeding more than 350 students and sponsors for lunch, and after lunch we managed to move a table from the venue to a free location.



Dead end

As easy as it sounds to draw some strokes, here we got stuck by the fact that all the visualisation in the J4K samples was done in OpenGL. Even though the GL object was accessible, one had to dig into the library to see how GL was set up internally.

After we couldn’t find any option to record sound from the camera, which was definitely required for the project, we decided to dismiss everything that we had done so far and switch from Java to C#. The people who have implemented the Java binding definitely did a great job, but with the Xbox logo on it the Kinect definitely finds more support in the .NET community. Installing and setting up Visual Studio and Xamarin Studios for our Windows and Macs respectively took some time, after the fresh start 8 hours in.


Fresh start

Starting with code from the colour basics sample shipped with Kinect SDK, we quickly caught up to the previous state. As convenient as in J4K, the mapping from 3D to 2D coordinates was already implemented. We started with drawing circles at the position of the tracked hands to verify that both tracking and coordinate mapping were working accurately.


A busy night

Apart from the spiritual sleeping class that was supposed to replace four hours of sleep in just 20 minutes, we all made it through the night. With the constant activity at the tables all around us hours flew by quickly until we finally put all parts together as dawn approached


Surprising turn of events

The closing ceremony was just as glamorous as the opening one. Over the first half an hour, both sponsors and audience were entertained by some really creative pitches from other teams. Unfortunately, mid-way through the ceremony, it was also time to say goodbye to our friend Benedikt, who travelled from Germany in order to attend the hackathon. What we hadn’t anticipated for was being called down for the finals presentation, shortly after Benedikt left. Once it had finally sunk in, we realised that all the hardware was already given back! Frantically, James made a dash back to the stalls to fetch the Kinect and wires. In the meantime, the rest of us waited and talked amongst ourselves about the exhilarating turn of events. We were surprised yet again, once our and the remaining groups’ presentations finished, that we won Microsoft’s prize for best use of its Cognitive Services! Microsoft sponsors also gave us a huge box of tech goodies, to share amongst ourselves. This hackathon has been a highlight for us all… and we want to thank DocSoc and the sponsors for an unforgettable weekend!




· We will never forget the Chinese characters for “forest”, “cat”, and “dog”, which we extensively used for testing.

· We gained a lot of experience to use Cognitive Services and the Kinect SDK as well as how pair programming can improve work efficiency.

· Hackathons are a great opportunity to meet like-minded people.

Project Information URL: