Are you ready to design for AR / MR and Spatial Computing?

with No Comments

With the rise of Augmented-Reality SDKs (Apple’s ARKit & Google’s ARCore), the new Mixed-Reality Glasses from Microsoft HoloLens, and Magic-Leap, it’s time to step-into an entirely new world of apps and experiences.
And for the UX/UI Designers and the Developers among us, it’s also a great opportunity to create productive apps for the Spatial Computing! So if Cross-Platform devices (Mobile, Tablets, Desktop, and IoT) is not enough for you, Then maybe you should dive into the future!

Step into Augmented-Reality (AR)

Just like many creative developers and designers out there, my first experience with AR was back in 2009–2010 when AR was so primitive, we had to use some sort of markers in order to make it work…
And then in 2013, when I tried Google Glass for the first time, I was so excited because it looked like the beginning of AR as it should be — on your eyes (as Glasses), and not in your hands (like a Smartphone). But it wasn’t perfect yet, although I could see the potential in such product for contextual HUD when driving, riding a bike, or when navigating the city on foot. So It wasn’t true AR/MR experience, at least not for me…

From AR to Mixed-Reality (MR)

Few years later, when I joined Microsoft and tried The HoloLens, I was amazed, and I was also convinced that it’s the future of AR! For me, it was the leap from AR to MR. Because MR is more immersive than AR. in MR, virtual objects can interact with real-world objects, it doesn’t blocks your view (like in VR), and you don’t have to look up (like in Google Glass) in order to see AR/MR content on top of your real-world view.

Control your Tech & IoT devices like a Jedi

And now with the Magic Leap hype, looks like it’s not that revolutionary compared to HoloLens, but I do like what they are doing for creators, and I do enjoy the competition — Hopefully it will help the AR/MR/XR tech to go faster, and evolve exponentially.

What About Apple

Well, I’m not sure if Apple is working on AR/MR Glasses, but if they are, then they’ll have a huge advantage over the competitors, thanks to their ARKit. Developers are already creating so many useful, productive, and innovative AR apps for iOS and Apple’s eco-system. And I believe that Apple will make it easier for Devs to transform their current AR apps from iOS to Apple’s future glasses (if they are developing it).

And judging by their career website, looks like they are recruiting relevant talents (such as, Optical Engineers). Btw not only at the Cupertino HQ, but also at the Israeli R&D center in Herzliya (near Microsoft R&D center). And just like Microsoft Israel, I won’t be surprised if Apple is developing it in Israel, because Microsoft HoloLens was developed in Israel. As you know, Apple has acquired few Israeli Startups in the recent years (The Startup Nation), including PrimeSense, Anobit, etc. So who knows ???? …?

Prepare for the future, now!

For years, I had so many ideas for AR/MR apps, but now is the time! It’s not hard and challenging as it used to be in the past, thanks to AR/MR guidelines, tutorials, and testimonials, you can dive in easily and start transforming your ideas into AR/MR apps. And when you’ve got a good use-case, and an useful idea, it’s time to write them down and pick few good ones before prototyping. So, in the past few months, I’ve been working on few prototypes for AR & MR apps, particularly productive apps that can enhance our capabilities and experiences.

Perhaps, even upgrade some of our human capabilities when it comes to knowledge, skills, Hybrid-thinking. After all, Elon Musk’s Neurolink is just over the corner, and I believe that combining MR with AI will upgrade our capabilities (just like Cyborgs).

I made few prototypes and use-cases (more to come) :

Some are in development, so I’m keeping them to myself (for now ????).

1. Study with AR/MR glasses

This AR/MR App prototype for students is how I believe we should read books. And also study, and save information in a productive way that will allow us to access that information whenever we need it, by using keywords and Hash-tags. In the future we will have AR/MR glasses (or AR contact lenses) and it will be only natural to access information and knowledge, without having to remember where you’ve read about it, or even without Googling it. This will change the education system, and how we remember and access the things we’ve learned / read. Just like having super powers…


2. Sketch with Mixed Reality glasses

This is how creative people can be productive with AR/MR glasses (or AR contact lenses). Sketch on paper, save it to the cloud, or locally to your glasses. Duplicate, paste, scale, transform, generate, and do things faster without distinguishing between the real-world elements and the virtual world elements. This will change the way creative people do their work. And for the UX designers out there, imagine how cool it will be to get Axure / Sketch features in the real world. Be creative and productive, Just like a wizard.

3. Control IoT devices with a AR/MR Glasses

This is how I believe we should control our Smart-Home IoT Devices. Not with remote controls and Smartphones, but with your hands, just like a Jedi! In the future we will have AR/MR glasses (or AR contact lenses) and it will be only natural to control everything just like having telekinesis capabilities!

4. Control Lights and AC with a AR/MR Glasses

Mixed Reality and IoT light switch. Because who needs physical switches when you can interact with IoT and Smart Home devices like a Jedi… ???????? I made this with ARCore and WiARframe tools in half an hour.


5. CyberSecurity App for AR/XR/VR, Wearables and IoT devices

SmartProtect for AR/MR/XR/VR, Spatial Computing, Wearables and IoT devices! Because we need better security and privacy in these new tech-gadgets. They are full of Cameras, Microphones, Sensors, Eye-tracking, and personal data miners. SP puts you in control on HoloLens & MagicLeap.


6. Jogging and watching videos or lectures

I always wanted to stay productive and even learn something new while jogging, but with a smartphone in my hand it’s too dangerous! But soon, it will be possible to do just that, thanks to AR / MR Glasses!
It’s been a while since my last AR app.

7. Control your Tesla Model S with your AR/MR Glasses

Another quick concept: Control your IoT devices like a Jedi! This time Tesla’s Smart key and IoT connectivity with AR/MR glasses. Now you can remotely watch your internal dash-cam , just like you are sitting in the car. And with Tesla’s SUMMON feature, one day you’ll be able to drive your car remotely!



8. Re-live your memories and moments

Re-live and experience photo albums in AR/MR: Find out location related photos from your friends and family members.


9. Control games & apps with your head, face, and voice

Few years ago, I had this idea to develop a game that you control with your head. But why develop a new game when it’s possible to apply this UX to all games & apps? Thanks to AR MR (ARKit & ARCore) SDKs now it’s doable (even with our face)! The bugs are because it’s still WIP.



By the way, You are more than welcome to share your ideas, concepts, and Apps you’ve designed / developed for AR, MR, and Spatial Computing… Let’s see if we can move things faster, and maybe Spatial Computing will become our reality even sooner than we think.


How to start with AR/MR?


Leave a Reply