Avatars 2.0: ready for Mixed Reality

Nintendo was the first of the big three video game companies to have an avatar system on the market. Where Nintendo lacked in online services, they excelled in social and party games and the Mii avatars were used in games like Wii Sports, Wii Play, and Wii Fit in order to provide a consistent multiplayer experience across games.

Microsoft’s take on avatars were first added to the Xbox platform a couple of years later in 2008. Xbox Live Avatars were created by Rare as part of a wide reaching revamp of the user experience labelled as NXE (New Xbox Experience). The NXE brought aspects from the Media Center (and Metro) into the dashboard and it paved the way for the Xbox experience we know today.

Unlike Nintendo’s early attempts at connecting friends (12-digit numbers) there was already a well-established community on Xbox Live and the new avatars were quickly integrated into basic features like the friends list, but it was no coincidence that Microsoft’s avatar system came just before the Kinect came on the market.

Many of the games for the Kinect acted as direct competitors to Wii games and avatars were used in Wii-competitor games like Kinect Sport, as well as more online focused games like 1 vs. 100.

Arguably, the Kinect seems to have died with the Xbox One and the original avatar system has been left exactly as it was. Today, you get the same functionally we got ten years ago, essentially.

Existing avatars are pretty basic and there is a limited set of skin tones and hair styles. My avatar wears a hat… because there isn’t the right kind of bald, for example.

Fast forward about ten years and Microsoft is gearing up to launch a huge upgrade to their avatar platform and this time it’s coming to Windows first.

Watch the new Xbox Avatars announcement

These new avatars look incredible and I don’t think it’s a coincidence that they are revamping their new avatar system at a time when Virtual Reality and Mixed Reality are starting to become a big part of Windows and Xbox.

We’ve seen examples of abstract avatars used as part of the Fluent Design System materials as well as the original introduction to the Windows Mixed Reality experience.

More recently, we have also seen less abstract representations. The examples above use live telepresence with Kinect and a basic scanned 3D representation.

You can easily see how these new avatars will fit right in into this spectrum of available avatars, but that’s not where it ends.

The new avatar system allows for a previously unseen amount of customisation and seem to be more human focused than anything we’ve seen before.

Human beings are most definitely a spectrum – we come in all ranges of sizes, genders, abilities, and conditions (temporary or otherwise). Having no choice on the number of limbs and only 17 choices of facial hair just doesn’t represent the beautiful range we have in reality.

You want to wear a floral dress?
No problem – Microsoft say there are no restrictions based on gender.

You want to have pink-but-slightly-purple hair?
No problem – Microsoft say there will be a free range of colour selection.

While I can’t find evidence that Microsoft has explicitly stated that these new avatars will also be used for Windows Mixed Reality, I think the very inclusive nature of the work they’ve done just proves that they understand the problem and are trying to solve it.

that the new Xbox avatars have been added in advance of a mixed reality push.

These new avatars have been created in Unity, which is one of the favourite development platforms for Windows Mixed Reality development.

The question of how someone wants to display themselves in Virtual Reality is an interesting one. Some prefer to see controllers floating in mid-air, others prefer to see renderings of arms.

Hopefully a range of abstract, realistic, and playful avatars will provide people with the choice they need to express themselves when using Windows Mixed Reality.

One thing is for certain: these new avatars are brilliant and I can’t wait to see what we can do with them.

Introducing Motion Controllers for Windows Mixed Reality

Microsoft have been using gestures like the air tap and bloom when interacting with the HoloLens, but when you’re in a fully occluded VR world, you need to be able to interact with your world without seeing your hands. This is done by using motion controllers, as seen by companies like HTC and Oculus.

Now it’s Microsoft’s turn to show how users are going to interact with the experience provided by Windows Mixed Reality. They are coming later this year and I can’t wait to get my hands on them.

Microsoft have a long history of creating new types of hardware to provide a consistent experience across third party devices. The original Microsoft Mouse was released alongside Windows Word in order to provide a way for users to move their cursor. Like the mouse it is in Microsoft’s interest to allow third parties to create their own motion controllers, but I expect that they’ll all be compatible and have the same technology inside.

Interestingly it seems that Microsoft has decided that they do not need to make these controllers work with the HoloLens – at least not with the existing version. The HoloLens hasn’t seen much in the way of software updates recently, so I won’t rule out the chances of them adding it in the future, but I get the feeling that Microsoft want people to use their hands for that device.

In my opinion we need these controllers for the HoloLens too – the Clicker is not enough… but that’s a story for another time. How do these new controllers work?

Optical Tracking

All of the Windows Mixed Reality headsets provide inside-out tracking – this means that each device will have the sensors required to track the world in the head-mounted display without any additional sensors or tracking devices around the room.

The new Motion Controllers take advantage of this technology to provide six degrees of freedom without any extra wires or mess. This also reduces the complexity of the controller, allowing for a complete tracking solution without being too expensive.

It’s worth mentioning that I assume there is some additional motion tracking in the controllers (so you can put it behind your back, for example), but the truly accurate measurements will be done optically when the headset have a line of sight to the lights around the controller.

Buttons

So what kinds of controls can we expect to have?

  • Windows buttons
  • Menu buttons
  • Trigger buttons
  • Grab buttons
  • Analogue thumbsticks
  • Trackpad surface

Currently, it’s unclear if the trigger and grab buttons are analogue or digital. Analogue buttons would enable the user to gently grab items as well as provide a wide range of trigger actions, much like accelerating in Forza when using an Xbox One controller.

I’m also super interested to know how well the trackpad surface can be used. It seems to have the ability to click, so it can be used much like a primary button too.

Watch the introduction video and see for yourself!

What is Mixed Reality?

I’ve not created a new category for my website for years – but there’s something that I have spending a lot of time working and I need to start writing about: mixed reality.

Mixed reality is a broad umbrella of technologies which enable a real and virtual worlds to be blended together to create a single environment where physical and digital objects interact with each other.

I remember when I was in school I would imagine what it would be like to have a “heads up display” all the time. I’d imagine information that I’d see about people and surroundings, interactive maps giving me directions, reminders about errands I need to run, all sorts of things.

Thanks to science fiction there was no doubt in my mind that it was an inevitability, and it would happen in my life time – but I didn’t really know how or when it would happen.

Fortaleza Leak

Back in 2012 a very interesting image appeared on the internet. It was from an Xbox presentation which mentioned something called Fortaleza Glasses. Rumours of Project Fortaleza had been going for a while, and I followed the subject with great interest.

Linked to Alex Kipman, the creator of the Kinect, it was thought that these glasses would be Microsoft’s move to leap ahead of virtual reality and into some new world of augmented reality. I hope so much that it was true…

HoloLens

..and I remember how amazed I was when I found out it was true.

HoloLens at Home

Well, over the last six months I have not only tried the HoloLens, but spend many hours using it in both professionally and in my home. I’ve been learning how it works, how to develop software for it, and most importantly; I’ve been learning about what it means for the future of software.

This is a whole new kind of user experience, and I believe that mixed reality is the future of how humans will interact with software.

Actiongram

But mixed reality is about experience, not technology. When I say that I believe it is the future of how humans will interact with software – I am not saying that everyone will wear headsets. It won’t be long until there are other ways to project digital items into the physical world.

It breaks down the walls between physical and virtual reality – today this means vision and sound, but the concept of mixed reality goes beyond those senses. As a software architect it excites me to think about the user experiences which everyone in the augmented/virtual/mixed reality community is going to help shape, and I’m to write about what I learn as I go.

Now that I have experienced mixed reality, there’s no going back.

It’s real, it’s getting better all the time, and it’s here to stay.