How Mixed Reality Will Change The World

in technology •  2 years ago  (edited)


The star of this article will be a little known (at least for the not so technical engaged ones) startup company from the US. But I'll come back to this later. First let me sum up the past few years of development and where we are today.

Virtual Reality

Since 2013, when the first developer kit for the Oculus Rift was released, a big hype around Virtual Reality (VR) evolved. Everyone who had a chance to put on this bulky head-mounted display was absolutely stunned. Despite the very bad resolution of only 640×800 pixels, it was an experience like no other before and it was hotly tipped as THE game changer in the entertainment industry, at least. The whole tech industry was scared up and it didn't take long to realize that there is an new market emerging.

Very soon a variety of competitors have joined the game, all with slightly different approaches, accelerating the VR evolution. Today VR has already fully established itself in the consumer market.







All these technologies do a pretty good job at tricking your brain and making you almost believe you are in a completely different space. But this also comes with a few downsides. First of all the virtual world doesn't know about the real world. So jumping through a beautiful green and peaceful meadow in virtual reality, chasing butterflies and enjoying the twittering of birds... could end in a sudden collision with something that feels pretty real, like a wall... or you might want to lean against something that isn't there. Funny for spectators, painful for you.

HTC's way of dealing with this is to use two sensors in two opposite corners of a room. With a little bit of calibration the Vive knows the boundaries of your room and displays an alarming gutter in the virtual world if you are about to leave them. This allows you to walk and jump freely and windmill your arms (like Google's AI does) in a virtual space of up to round about 30 m² without fearing the dangerous physical reality around you. Your freedom is only limited by the cables running from your head down to your expensive high end gaming computer. Fortunately wireless solutions are on the way.

But regardless of how good this technology might get in the future, it will always be centered around special use cases like entertainment or industry applications. Of course this is an amazing technology and we've actually been waiting for this at least since Nintendo's Virtual Boy (which still has it's own website). But it will never change the way we live like the internet and the smartphone did.

If you ask me, VR's "little brother" has far more potential to accomplish this.

Augmented Reality

Augmented Reality (AR) has proven it's value since the early 1990s when the US Air Force developed it's first real-world application. Like often in technology, the military is the driving force. Even the Oculus Rift founder Palmer Lucky has some military related background.


In contrast to VR, which replaces your entire perspective on the physical world with a virtual one, AR is merely a digital overlay, seen through a pair of glasses or through your smartphone display utilizing the camera.

About five years ago I was absolutely fascinated by an app called Augment. I remember a sunny afternoon, sitting in a beer garden and showing my friends a skeleton dancing on the table... on which their jaws dropped too.
The app is meant for commercial use but also did a good job in terms of entertainment back then. It's still free and available for download from Google Play and iTunes App Store.

I think the first really popular app that integrated augmented reality actually was Pokémon Go. At that time I hated my smartphone for not having a gyroscope and it kept telling me that the augmented reality feature of the game won't work. Yes Pokémon Go once was one of my guilty pleasures.

But obviously Niantic, the developers behind Pokémon Go, did something right. They combined the virtual and the physical world in a way that it was accessible for the masses and everyone could have fun with it. In fact they did this before with the predecessor of Pokémon Go, called Ingress. It's exactly the same game concept, only without the much-loved poket monsters and the AR feature.

Pokémon Go (Source) and Ingress (Source)

Anyway... merging the digital and physical world will be one of the most important and live changing technological advancements of the future. The problem with AR is that it lacks an understanding of the surrounding environment in order to adapt to it to feel more natural. This is where Microsoft enters the stage. And no, the term "startup" I mentioned at the beginning refers to yet another company. We'll get there soon. ;)

Mixed Reality

Mixed Reality (MR) is really just a more sophisticated version of AR.

With the help of advanced AR technology (e.g. adding computer vision and object recognition) the information about the surrounding real world of the user becomes interactive and digitally manipulable.
Wikipedia - Augmented reality

This is then called "Mixed Reality". In January 2015 Microsoft revealed the HoloLens and Windows Mixed Reality. I'm not a big Microsoft fan but this can definitely be considered as a first serious foretaste of what our lives will look like in the future.



The HoloLens bundles a variety of technology that's necessary to get a decent understanding of the world around it, like an accelerometer, a gyroscope, cameras, light and depth sensors and microphones. Together with a special MR-version of Windows 10 it can project user interfaces anywhere in the room, in the form of holograms. Everything is controlled with a few simple hand gestures and thank god... there aren't any cables. One of my favorite YouTube channels Computerphile did a little demonstration back in march this year. Another nice video is this one from ColdFusion.

Both point out the major downside of the HoloLens: It's rather small field of view.


Also, starting at $3,000 it is still very expensive and it also looks very bulky and futuristic, which I like but... it's nothing you want to wear in your everyday life as a replacement for your smartphone. But once the technology becomes more mature in terms of functionality and design, at an affordable price, this will be the tipping point where mass adoption by regular persons gets started. And this moment might be closer than you think.


So now we can finally talk about this little known startup company called Magic Leap. The company was founded by Rony Abovitz in 2010, but the first time they were recognized by a broader audience was in 2015 when they raised more than $540 million from companies like Google and Qualcomm. Alibaba joined in early 2016 with almost $800 million. If such companies invest so much money in a technology there must be something to it.

Their original concept video from early 2015 appeared to be pure science fiction but in the meanwhile there is a growing body of opinion that their technology is in fact as impressive as shown in the video or that at least it comes very close.

They are even cooperating with Lucasfilm to bring content from the Star Wars universe to their platform, which might be an indication that their technology could be indeed something we normally only expect to see in Star Wars movies.

But what makes it so superior to what Microsoft and others do? How does it work? Well, they haven't revealed any details yet but luckily there was a diligent enthusiast who dived into all available information like talks and patents to draw a picture of what we can expect.

How Magic Leap Works - From Field of View to GPU

Basically we can expect the device to be split into two parts. A pocket device that does all the computing and a pair of glasses with sensors providing information about the surroundings, both connected via cable. So instead of carrying your smartphone around you have this device of similar size in your pocket but you don't need to dig it up because you have the interface right on your eyes all the time or maybe hanging around your neck or in your shirt pocket like any other glasses. This is the key difference compared to something like the HoloLens or Google Glass. You can actually wear it in public without drawing much attention, which is very important for social acceptance of this new technology.

The best about this however, is the groundbreaking display technology. It uses optical fiber cables to project the augmentations directly onto your retina, with a field of view that is by far bigger than the one in the HoloLens. The latter can be described as looking at a 15" screen from normal distance while the magic leap glasses are supposed to fill almost your entire field of view.

They want to produce a field of view of 120° by 80°. This would be a larger field of view than the oculus rift and at far greater resolution. So have they achieved this? It is hard to say but the patent does give us some numbers to work with bearing in mind these are over 3 years old and they have likely improved the technology even more in the mean time.
How Magic Leap Works - From Field of View to GPU

It's obvious that they are trying hard to make it an actual "magic leap" in AR/MR technology. The big question is...

When and how much?

You might have guessed it already... there is nothing but rumors. Anticipated price is around $1000 and a release could be in early 2018 or even later this year. At least a final reveal of the product is likely to happen very soon. The Time Magazine claims to have more solid information but unfortunately you have to pay for it.

However, it's definitely safe to say that they are really passionate about their product and the odds are in favor of at least something to happen this year.

2017 will be a big year for Magic Leap. Enjoy the ride with us – it will be fun. Magic Leap is for the dreamer, the artist, and the wide-eyed kid within us all. But what we are building is no longer just a dream. It is very real, and we are way past the "is it possible stage". We are not about building cool prototypes. We are scaling up so we can manufacture hundreds of thousands of systems, and then millions. That requires a level of perfection, testing, and attention to detail by determined professionals. We have made something that is small, mobile, powerful, and we think pretty cool.
Creativity matters. Rony Abovitz - January 1st, 2017

I think we are just at the very beginning of digitization. Think of all the possibilities this technology offers. You can enjoy all your digital helpers without being distracted from the real world around you. Physical objects can be part of a digital experiences and vice versa. Most workplaces will be completely digitized while bringing the physical world more into focus and all physical work will be enriched with helpful real-time information to make lives much easier. And if you want to watch a movie you just draw the screen to a wall or in mid-air, everywhere. Skyping with your loved ones will be like sitting next to each other.

In 2006 the first iPhone was released. Ten years later we can't imagine a world without our little supercomputers in our pockets anymore. The smartphone affected almost every aspect of our lives by revolutionizing the way we communicate and interact with each other, how we read the news, work, play, enjoy entertainment or even how we keep an eye on our health and so on. I strongly believe that such a rapid evolution can happen again very soon. Another ten years from now we could already live in a totally different reality again, a mixed reality, where lines between the digital and the physical world are completely blurred. I'm just waiting for the day when Apple enters the stage and says: "Look, these are our smart glasses and you only need your iPhone 10 for it to work!" Personally I hope that companies like Magic Leap will enter this market on time to rock the boat.

So keep an eye on Magic Leap and the MR market!

Thanks for reading and please share your thoughts with me in the comments!

Here are three more videos that I watched while diving into the topic.

Post Image Source

Thank you for your attention!

Follow for more on

Read about me and my content.

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

magicleap Magic Leap tweeted @ 16 Jun 2016 - 14:19 UTC

We’re teaming up with #Lucasfilm to bring the force to the real world.…

Disclaimer: I am just a bot trying to be helpful.

Nice work, i hope you grow fast on steemit :)

Nice Post!
Thanks for sharing this.

Thanks for your feedback and your upvote! :)
It was quite some work... but also fun to write.

Hi @mkt! Great article! Found it through your reply to @jerrybanfield !
I just started a curated library post around the concept of the Idea Economy. From your writing here I think you'd be able to add very good value and might have some articles that would fit in with the curation goal. If you have any posts that you think fit, please submit them in the comments and I'll happily include it in the next edition of my library post.
Please mention though where you think it fits in and why someone should read your article. Keep up the good writing !
Post is here:

I almost missed this comment due to the missing notifications on steemit.... anyway. Thanks for your feedback! I'll follow you to be up-to-date on your curation project and maybe I'll commit an article in the future. Thanks for the info! :)

they don't have to be new either! I am aiming to disregard the whole upvoting game and focus on good authors and content!