Science and Tech Event Blog

Arlene Kalem's blog about all the tech and science stuff she gets herself into. Browse the archive by month or scroll down to search by tag. 

HACK THE BRAIN WOOOOO

Wow, what a crazy awesome past two days! I don't think I've ever been so stressed and so exhilarated in such a short time span before.

I joined the g.tec team; they had two company people there to help, and then I had to find a group. We had four people to start with, but it ended up being just me and a guy called Dionatan Moura instead. Funnily enough, Dionatan was the guy who ran the agile workshop I went to months ago! Dionatan is programmer by day and a pizza-lover by night. 

Dionatan and I goofing around

Dionatan and I goofing around

After much debate, we decided to make a game controlled by brain waves. We first started by trying to get the intendix software to talk to Unity. In the end, this was our main problem. Thankfully Dionatan is a java guru and was able to figure out C# pretty quickly; he worked a lot with the people from g.tec to figure out how how to get the systems to communicate in Visual Studio. It took us a long time, basically the entire first day, where all of us sat stumped at the computer and with the intendix guru's fingers fiercely flying over the keyboard. I had the dubious pleasure of being connected to the computer the entire time via the BCI tech. I had a lot of gel to wash out of my hair that night. 

When we realized how long it was taking just to get the BCI tech and VS to talk, we decided to use a game I had already built, my crappy but simple 2D game Stardust, instead of trying to build a new one. We had gotten Visual studio to receive data, but we had to tell Unity what the heck to do with it. I spent that night reprogramming the physics and commands of the game. 

I spent two days in this cap. We became best friends. I was sad to see it go. 

I spent two days in this cap. We became best friends. I was sad to see it go. 

So basically the solution was to send data from the intendix software to VS over the ip port; the incoming data was in a byte array, so we had to turn the byte array into a string. Essentially for each image, the intedix software created a unique string. We the created an enum where each string represented a command; receiving that command would trigger game object actions in another script. 

We ended up having three total commands: Up, Down, Shoot. 

TLDR: We used a skull cap to measure brain waves and used a flickering interface to generate a response in the brain to an image, which then triggered an action in the unity game engine. So for example we had an image of an up arrow, and by looking at it, the player could make the game object move up. 

You can watch a video of our presentation and see the game in action below. 

One of the projects developed during Hack the Brain Dublin, part of the BrainHack project and the EU Horizon 2020 program. Dionatan Moura and Arlene Kalem develop a game in the Unity engine controlled by brainwaves by using the intendiX Speller software provided by g.tec.

BCI Workshop

The day before Hack the Brain, I attended the workshop on BCI tech hosted by g.tec to get a better idea of what is going on (alas, I am not a brain scientist). I admit I wasn't too terribly impressed at first. The presentation was delayed over 30 minutes (not uncommon in Ireland), and the projector screen didn't work at first, and then the presenter's mic was super quiet. 

However, it eventually paid off because there was a wicked awesome demo of the intendix speller software. We also got the low-down on different types of headgear; g.tec uses a cap, with the nodes affixed by gel (gets in your hair - fun!). Essentially, what the speller software looks for is the p300 brainwave and memorizes its formation for different letters. 

First the software has to be trained. Basically, it will pick a letter and the user has to concentrate on that letter for at least four "flashes". The software will alternate between flashing the letter and flashing a picture of a celebrity. The software memorizes the shape of the brainwave each time the letter flashes onscreen. Apparently flashing to familiar faces (like celebrities) instead of flashing to a black screen helps the process. 

Then, after being calibrated, when the user concentrates on a letter of their choice, the software recognizes which letter by the shape of the P300 brainwave. Thus, the user is able to spells words just by thinking of the letters.  Here's a very short video of someone using the software. It's pretty neat! Look at how fast the flashes go!

Hack the Brain Pre-Event

Hack the Brain is coming to Dublin! It's a global event that invites artists, scientists and the general public to work with brain interface technology and create something fantastic. The selected projects for BrainHack Dublin put on a presentation. I signed up to participate and will get to join one of these teams. There is a team from the company g.tec, and they want to experiment with using Unity and with BCI tech such as the intendix spells software! I think I'll join them. There are other projects from indie developers and researchers and the like but they're not using Unity (one team is using Unreal but I do not know C++. Must take a class in it!). 

It's going to be a 48 hour event in June, hosted at the Science Gallery on Trinity's campus. Convenient for me!