Thesis Update#15 - Science Gallery Exhibition

And finally, here we are full circle. My first post in this blog was about the previous IDM class's projects. Now here I am showcasing my own. It's essentially the end of my degree, the end of my time here in Ireland, and finally the end of this bloody project. I'm not even sure what happened this weekend.

We were open to the public on Saturday and Sunday at the Science Gallery, a supremely nice venue that is very close to my heart after spending so much time in there during the hackathon. Then on Monday morning we were graded by the professors and gave our final presentation. On Tuesday, we had one last showcase, this time a smaller one just for all the graduating computer science students (we brought our projects, the rest of the students brought their super smart sounding poster boards...so basically ours were cooler). 

I did a lot of programming on the fly, including providing more feedback on the buttons. I was able to make them change color when pressed, and back again on Clear Input and on the start of a new challenge. It didn't work the first time and it was embarrassing, but I got it to work in the end. I admit I wasn't in top form - stressed and ill (yay mono!) and a little annoyed at everyone. 

But we did it. We finished. And I met Ken Perlin. My idol who worked on Tron and created that sexy algorithm. He was our external examiner. Naturally, our project kept breaking when he tried to use it (3 bluetooth connections didn't always like to connect). Embarrassing, but at the end it was all worth it. We went out to a pub and I sat right next to him and talked to him for like three or more hours. He said he thought our project was the most interesting because we broke the most rules; we didn't do anything safe, nothing guaranteed to work, nothing standard like a website. 

So that felt real good. And then we shared a pint and I was satisfied with life. 

I'd completely redo the way I handled the project if I had a choice, but damn did I learn a lot! And really, that makes me a smarter leader and developer for the next project. I made a lot of mistakes, but I learned a ton. That certified unity developer certificate is well earned. I'll probably make a separate post for pictures and videos, but most of it on the KORE main project page on my website. I'm not sure I have it in me to look at this project again any time soon, even if it's just to post pictures and videos haha. Talk about burnout! But I accomplished what I wanted. I wanted to do Unity and VR, and for three glorious months my only job was to learn that (and more! hello unexpected bluetooth and arduino). 

Thesis Update #14 - 3D Camp/Irish VR - as an exhibitor!!

On Thursday, the 24, we did a user test at 3DCamp. 3DCamp is a meet-up group that is hosted at Work Day. They meet every three or four months and showcase tons of projects on a variety of platforms, even some of the newer and unique ones like the SmartHelmet from Daqri or the HoloLens from Microsoft. There are a lot of projects from local companies, students, and entrepreneurs. We got to talk a bit about the project and then did our first full dress rehearsal. It was cool to finally be on the other side of things, as usually I'm a visitor. 

Needless to say a lot of things weren’t working and a lot of things went wrong. It was a good experience.

We have decided to combine the panel and reference book. It was taking people far too long to find the challenge and then find the panel. So we are going to put the panel page right beside the challenge solution page.

Additionally, people were having a hard time knowing what to communicate to the other players. We are going to make a small webpage or app that has instructions and role descriptions that all the players can look through beforehand. That way they will know the necessary steps, and see photos of the environment and sensors before they begin to play. We’re going try to keep the instructions short, simple, and to the point.

It was also a bit difficult for people to read out the challenge, and then for the second player to match up that challenge to something in the book. We need more of a definite, hard link between what appears on the podium and what is in the book. Eva had the idea to put the panel symbols up on the podium, and then the player one would communicate that to player 2. Player 2 can probably find that easier in the book. I think it eliminates quite a bit of the story, but for the sake of playability and clarity, this change probably needs to happen.

I attempted to enact some of those changes over the weekend but unfortunately fell ill and had to go to the doctor. I have contracted a viral infection (yay mono) and a fungal infection (yay Candida overgrowth). My weakened immune system couldn’t just wait one more week, could it?

Thesis Update #13

I redid some of the UI to make it more intuitive. I made the player spawn in front of the globe and brought the UI in a curve around the player’s spawning point instead of around the globe. This is because people were having a hard time finding information on the centre podium.

I also programmed in all of the challenges and icons that spawn according to the challenge that is randomly selected. We also now have audio files that Shreya recorded with a classmate. They will play when a challenge is announced. The centre podium also displays how many challenges they have left to solve, how many they got right, and if they entered the button press sequence correctly.

In addition, I also changed the UI that follows the player. The orb idea was cute, but I couldn’t think of a way to represent Clear Last Input and no one else could either. So I decided to use the panels from the centre podium. Those large panels actually consist of two separate, smaller objects. I duplicated those smaller objects to form the three prongs of the UI below. Then I put a square in the middle to be Clear Last Input.

I used radial displays because I thought they fit the shape of the UI better than the standard health bar display. If I had more time, I probably could stylize them a bit more – maybe have a parchment background instead of the white circle or something. But the panel symbols are large, blocky and white, and this sort of matches that as well. Plus, the UI is readable and colorful and in the confusion of running around trying to find the right panel, I think it’s a nice little beacon of clarity.

Sometimes, though it gets in the way of the user’s view or path. I’ve still got some tweaking to do with that.

I also programmed the recticle so it changes colors if it hovers over an interactive object and if the user clicks on an object. I hope that will give enough feedback.

Thesis Update #12

As stated in an earlier blog post, Karl and I decided to divide KORE up into nine parts across nine different scenes in order to optimize the game for mobile. Unfortunately, we weren't able to get the Bluetooth connection working properly over the multiple scenes. While the connection would hold, for some reason the information from the button presses wasn't going through.

We had to come up with another plan. During our research, we came across something called occlusion culling, a feature in Unity that basically doesn't render anything the camera can't see, and also won't render anything that is blocked by another object. Karl had already modeled a fantastic set of of rooms, so we crammed all of them plus 64 panels and 320 buttons into one scene and tried it out with occlusion culling. We were able to create a text box that displayed our frame rate, which was really helpful in determining what features caused the frame rate to drop. Even with occlusion culling, the crystals caused a significant drop - I'm talking minus 10 - in frame rate. Thus, we've decided to use symbols instead of crystals to denote the individual rooms. This helps pull the idea together anyway, as it's very clear now how the rooms relate to the challenges inside.

However, we hit another snag. For some reason, the Bluetooth wasn't working at all in the new massive scene! We even replaced our scripts with older versions that worked, but still no dice. However, we still had one of our earliest versions of the project with Blue tooth working. So, we decided to import all of our assets from the new project into the old project where the Blue tooth was working. So I had a fun few hours doing that!

Additionally, I had just used copies of the Envy panels to do the occlusion culling test. I still had to fit out each room with panels, mix up the button patterns a bit, and give them each textures themed to the color of the room (while we were no longer using the crystals, we were still using the color scheme Eva created).

So essentially over the weekend I placed: 64 panels, 320 buttons (and textured them), placed around 12 lights and torches, and then also placed in the 64 unique symbol identifiers on each panel (plus took screenshots of those for Eva and the panel book). Then I programmed the main 16 panels and their buttons so they were clickable and had winnable patterns using the list script Karl and I had written about two months ago.

Lastly, for the user manual, I wrote 64 unique buttons press sequences.

And that is why I never want to see a panel or button ever again! It was a lot of work completely rebuilding KORE and then finishing all the fiddly details. We still have to make the game be able to run by itself (right now we push a button to instantiate a challenge - we will have to trigger it automatically in the game). While I was able to work a bit on the UI, we still have to time those up to the challenges as well.

A bit of good news, though. We did manage to get multiple blue tooth connections to the same scene! We couldn't use the Wii controllers due to various reasons, but we are able to have two extra arduino/Blue tooth connections to help fuel our accelerometer and gyroscope data for the heat and oxygen controls. So now we have to program those!

We are trying to do a user test on Friday, at least with the Envy room, to give us insight into how long it takes users to complete the challenges and how hard/easy it is for them to do so. In addition, we are hoping to include an introductory video. I wrote the script and Shreya filmed it about two weeks ago. Now that KORE is looking good, I can get some screencasts of it for the video.

Looking forward to finishing this project and never seeing it again :)

Thesis Update #11

This past week has been a tad bit frustrating for Karl and I. We managed to get the NFC tags working in one day, but the victories stopped there. With help from Alberto, we were able to set up networking between the tablet and android phone, with the tablet as the host. However, due to the weak and infrequent network connection, we were unable to sustain the service long enough to do full tests. We also quickly ran into another problem. Our test phone, the Nexus 5, could not keep up with our huge scene. The phone repeatedly overheated, sometimes froze, and was constantly lagging. In order to secure a stable frame rate, we had to delve into mobile optimization.

This is something we really should have looked into before starting to develop an idea and the assets. Knowing the limitations of mobile beforehand would have saved us as a lot of trouble and heartache later on. So yesterday and today, Karl and I made sure to have simple colliders instead of mesh colliders, made basically everything static, and redid assets that were polygon heavy. However, as our lights are triggered by the user, we have to have dynamic lights, which mobile phones are bad at handling. What we can do, however, is paint in the shadows beforehand using images and turn off real time shadows in unity. Additionally, we are using a lot of materials, so now we will simply be using the textures instead.

We have also decided to make a major change to how the game is laid out, which Karl and I will be testing over the weekend to be sure that it is viable design. Right now the KORE lab is a huge space with everything in it all rendering at once, which creates an insane amount of triangles and vertices. Karl did some research and discovered that asynchronous loading could save us in that area. To quote the Unity manual, asynchronous loading "allows you to create a completely streaming world where you constantly load and unload different parts of the world based on the player position, without any hiccups in game play".

Basically, we will be breaking the game up into nine different pieces. Instead of one major room, we would have a small main control room with eight doorways leading to different rooms, all of which are in different scenes. The game only loads the scene the user is in and the one they are headed into. This means fewer assets, scripts, and lights will be loaded at once, which will give us a greater performance boost, and will also let us keep some of our higher polygon models. Additionally, this helps organize the play experience for the user. The first thing they will see is the main UI as well as eight passages guarded by eight different types of crystals. Therefore, they will need to know the crystal first before they can choose a room. This will help guide the user through a more logical game progression, as we give them a place to start. With one huge room, it was a bit like sensory overload.

Karl described it best as a passage tomb, such as Newgrange. So over the weekend Karl and I will be making a small prototype to ensure it works as we hope it will. We will be modeling two separate rooms and fitting them out with all the required models, textures and scripts so we can test the design.

We also had a group meeting with out supervisor, and though we can't get a Vive (awwww), we can get a newer phone such as the Google Pixel or newly released Samsung S8, which both support Google Daydream. Having a newer phone that has been designed with VR in mind will definitely help us with game performance.