Introducing Motion Controllers for Windows Mixed Reality

Microsoft have been using gestures like the air tap and bloom when interacting with the HoloLens, but when you’re in a fully occluded VR world, you need to be able to interact with your world without seeing your hands. This is done by using motion controllers, as seen by companies like HTC and Oculus.

Now it’s Microsoft’s turn to show how users are going to interact with the experience provided by Windows Mixed Reality. They are coming later this year and I can’t wait to get my hands on them.

Microsoft have a long history of creating new types of hardware to provide a consistent experience across third party devices. The original Microsoft Mouse was released alongside Windows Word in order to provide a way for users to move their cursor. Like the mouse it is in Microsoft’s interest to allow third parties to create their own motion controllers, but I expect that they’ll all be compatible and have the same technology inside.

Interestingly it seems that Microsoft has decided that they do not need to make these controllers work with the HoloLens – at least not with the existing version. The HoloLens hasn’t seen much in the way of software updates recently, so I won’t rule out the chances of them adding it in the future, but I get the feeling that Microsoft want people to use their hands for that device.

In my opinion we need these controllers for the HoloLens too – the Clicker is not enough… but that’s a story for another time. How do these new controllers work?

Optical Tracking

All of the Windows Mixed Reality headsets provide inside-out tracking – this means that each device will have the sensors required to track the world in the head-mounted display without any additional sensors or tracking devices around the room.

The new Motion Controllers take advantage of this technology to provide six degrees of freedom without any extra wires or mess. This also reduces the complexity of the controller, allowing for a complete tracking solution without being too expensive.

It’s worth mentioning that I assume there is some additional motion tracking in the controllers (so you can put it behind your back, for example), but the truly accurate measurements will be done optically when the headset have a line of sight to the lights around the controller.

Buttons

So what kinds of controls can we expect to have?

  • Windows buttons
  • Menu buttons
  • Trigger buttons
  • Grab buttons
  • Analogue thumbsticks
  • Trackpad surface

Currently, it’s unclear if the trigger and grab buttons are analogue or digital. Analogue buttons would enable the user to gently grab items as well as provide a wide range of trigger actions, much like accelerating in Forza when using an Xbox One controller.

I’m also super interested to know how well the trackpad surface can be used. It seems to have the ability to click, so it can be used much like a primary button too.

Watch the introduction video and see for yourself!

Highlights from Build 2017

I tend to describe Microsoft’s Build conference as a bit like Christmas for developers who use Microsoft’s tools and technologies to build software. This year was no exception – and there was plenty to be excited about.

As per usual, there is a vast amount of content published on Channel 9, most of which I have not gone through yet, but here are some of the top announcements that interested me the most:

Microsoft’s democratised AI offerings continue to grow and improve customisation

Microsoft have been promoting their Cognitive Services for a while now, and they’ve been getting more and more robust over time, now with 29 services up and running and available for developers to use.

One of the most exciting additions this year is the trainable image services. Being able to train AI to spot certain attributes on images is something that can have a huge impact on some of the technologies I build professionally.

The addition of Cognitive Services Labs allows developers to try out more experimental AI services, including Project Prague, a gesture recognition service.

It’s also worth mentioning that Satya said that, as solution architects and software developers, we should take accountability for the algorithms and experience we produce. We should be building inclusive systems which help empower people – in a way that they can trust. I agree with him.

Azure Cosmos DB is a shiny new multi-model global scale data service

As well as bringing much needed MySQL and PostgreSQL service offerings to the cloud, Microsoft have also announced their latest home grown cloud-native database service, Cosmos DB.

As a software architect, having Cosmos DB will allow me to make much better choices about the consistency of data solutions I am designing without having to worry about indexes or where the data will rest at run time.

The global distribution of Cosmos makes it a lot easier to make ensure that the data is as geographically close as possible to the end user. It’s essentially an extension of Document DB, but allows for a multi-model interface: key-value, column family, graph, and document.

As Cosmos DB is built on the Document DB technologies, there is already an emulator which can be used locally at development time. For me, this is a must when choosing cloud technologies.

For me the timing of the Cosmost DB announcement is really great, as a planet scale database is something I’ve actively been looking at for a new project I’m working on. I’m looking forward to learning more about it.

New tools for Azure developers and administrators

Azure is becoming one of the most important assets that Microsoft has. It’s the centre of many of their initiatives including AI, IoT, microservices, and more. Their continued work to strengthen this platform has made it easier than ever for developers to get up and running with all of these new services through a coherent set of tools and development kits.

New tools like the Cloud Shell and the Azure Mobile App are part of this. Unfortunately for me, the PowerShell version of the Cloud Shell isn’t available yet, nor is the Windows version of the Mobile App. However, the improvements to the Azure CLI are most welcome. Under the covers the Cloud Shell uses the shiny new cross platform command line interface for Azure and is already logged in and configured, making it super easy to get up and running. I’m a huge fan.

We’re still missing an Azure desktop app though – I still think there’s value in having a version of the Azure portal that doesn’t require using a web browser. Using Electron is probably the best way for Microsoft to achieve this and I’m unsure why they’ve not already provided a desktop app.

A powerful new feature called Snapshot Debugger will integrate with Visual Studio to make debugging production easier than it ever has been. You can create snap-points on certain lines of code which will instruct Azure to collect information as the application is used. It’s very impressive, and doesn’t affect people using the production application in any way.

I’m keen to try this out but it seems like it is going to be a powerful new way to fix issues in production without the security risks involved in pulling production data to a developer’s local machine for debugging. Awareness of production data is a must for companies who use customer data, and tools like this will help with adherence to the Data Protection Act and security standards like the popular ISO 27001.

Microsoft has a new mantra

A clear message from Build 2017 was that developers shouldn’t be placing all of their business logic and intelligence inside Microsoft’s cloud infrastructure, instead they should be considering how devices on the edge of this cloud could be leveraged to improve the solution.

Intelligent Cloud and Intelligent Edge

Not only does this make more sense, but it’s also something that Microsoft is uniquely positioned to provide. As a long term supplier of back office / on premise software, they’ve already got a foot in the door of many companies data centres. Improvements to Azure Stack and Azure’s IoT offerings allow logic to be moved between Azure’s cloud, to on premise data centres, and even to embedded edge devices.

Azure IoT Edge. is an example of how logic can move between the cloud and edge devices through a single management infrastructure:

  • Run AI at the edge to reduce latency and allow for offline-scenarios
  • Perform analytics and proactive decisions at the edge
  • Move logic from cloud to edge at any time
  • Management of edge devices from a central location
  • Simplify development
  • Reduce bandwidth costs

While these tools are very interesting to me, I have a feeling we’re still a little way off. The innovations here are huge and not to be taken lightly and I expect more to come over the next few years.

Cortana and Bot Framework improvements

One of the more obvious changes is that Cortana has come out of the phone itself and she’s now coming to other devices like the Harman Kardon’s Invoke intelligent speaker. (Yes, this counts as an intelligent edge device!)

General improvements have been made around the Bot Framework too. It’s now easier than ever to use natural language for common actions like taking payments from users.

Cortana Skills have been created to better link Cortana with services built on the Bot Framework and Adaptive Cards make it easy to write interactive cards which work across all platforms.

These integration improvements aside, I’m not convinced Cortana herself is moving fast enough and I’ll have to write up some more of my thoughts in a follow up to last year’s thoughts.

Windows 10 Fall Creators Update

Aside from the stupid name, it looks like there has been a steady progression for the Windows 10 platform.

The update brings a number of much-anticipated features including a cross-device clipboard, pick up where you left off, OneDrive on demand sync, and much more.

One of the best new features was the timeline view, which shows previously used applications across multiple machines. I’m not sure how well this will work for me, so I’m looking forward to getting my hands on it so that I can try it out.

Interestingly, the addition of a few apps to the Windows Store have caused quite a commotion:

  • iTunes – a must-have for iPhone users will be coming to the Windows Store. I don’t use it, but I understand the gravity of what this means to users and the pressure it will apply to Google to bring their apps to the store too.

  • Linux – we’ve had Ubuntu for a year, but now the Windows Subsystem for Linux has been updated to include Fedora and SUSE. Who’d have thought it would be Microsoft to really bring Linux to the desktop?

One of the more impressive apps was Windows Story Remix (the video is worth watching!), which takes advantage of many of the platform and service offerings to include an impressive experience for users who want to create video content with their photos and videos. While this isn’t something I do very often I certainly appreciate how well Windows Story Remix has been executed.

The fall update also brings the long-awaited replacement for the Metro design language…

Fluent Design System 😍

Microsoft’s design system has had a rocky past due the company being forced to drop its “Metro” identity early on in life, and it has hobbled along with the less memorable “Microsoft Design Language” since before Windows 10’s introduction.

Finally, they’ve sorted themselves out and come up with a new name for their design language.

While it is an evolution of the existing Metro principals (see my previous rundown), the new design’s new direction takes into account five key areas:

  • Light
  • Depth
  • Motion
  • Material
  • Scale

Fluent Design is something that really interests me, so I’m going to write more about this in an upcoming post.

Developer Tools, New APIs and much more…

It’s no surprise that there have been a load of improvements around the developer tools and other services too:

  • Visual Studio 2017 for Mac
  • 3rd party integrations for Microsoft Teams
  • .NET Standard 2.0 and XAML Standard 1.0
  • Azure Functions Improvements
  • Much more…

Exciting times!

Project Scorpio: The Next Xbox

Last year, I upgraded from my Xbox 360 to an Xbox One S. At the time, I knew that “Project Scorpio” was going to be coming in late 2017, but the time was right for me and I wanted to move onto the Xbox One platform.

The fact the Xbox One S and “Project Scorpio” were announced at the same time was an interesting move. Game consoles aren’t usually announced so early, but this current generation (often called the 8th generation) of consoles is likely to be around longer than others.

Watch the Project Scorpio announcement

Both Sony and Microsoft have adopted the x86 processor architecture found in PCs, and while they’re still highly customised, the development of this common architecture is good for the console makers and the software developers alike.

We’ve already seen an updated PlayStation 4, so an updated Xbox just made sense and we’ll likely see more hardware refreshes in the future. I bet that games for Xbox One will continue to be developed and enjoyed even longer than the previous generation. The Xbox 360 stayed on the market for 11 years and its games can be enjoyed through backwards compatibility on the Xbox One today.

The message is strong

Microsoft has been very clear that “Project Scorpio” is a mid-generation refresh, but this time it’s a performance boost to the machine itself while remaining 100% compatible with the all of the Xbox One games and accessories currently on the market.

“The most powerful console ever.
Holiday 2017.” – Microsoft

They’ve also been clear that “Project Scorpio” has been designed for the fans who want the best. Microsoft stated that they wanted to make the most powerful console on the market – and it looks like they’ve achieved it.

A high-end version of the Xbox One

The performance updates on the machine itself are designed to enable 4K gaming and new VR experiences, though it is expected that existing Xbox One games will also see a general performance boost, even when displaying on 1080p televisions.

Even though it has been stated that there will be no games which will be exclusive to “Project Scorpio”, I have no doubt that there will be some games that will take advantage of the extra power and will be best experienced on the new machine.

Forza on Scorpio

Some existing Xbox One games (Gears of War Ultimate Edition, Forza Horizon 3) already include 4K assets, so the work to upgrade the games to work on high resolution “Project Scorpio” would be minimal. I wonder how many other games have already got high resolution graphics ready for 4K on day one.

Microsoft have really come together

One of the most impressive things about “Project Scorpio” is that it has been built with the full power of Microsoft behind it:

There’s no doubt that Microsoft is a hardware company and their expertise has also allowed for impressive cooling and performance tuning throughout the machine.

Scorpio

DirectX is now built in to the hardware. This is really impressive and means that the hardware has to do less work for games built using DirectX APIs.

Existing games have been profiled for performance and the telemetry of the software has gone into the design of actual silicon. This is a really interesting technique for Microsoft and may help direct performance improvements for their Azure cloud platform in the future.

“Project Scorpio” has been in the works for a while

I recently re-watched the original Xbox One announcement – it was really bad. They announced it just before E3 and had a focus on TV, entertainment, and the use of Kinect.

Since then, the management of the Xbox operation has changed and they’re now way more focused on the feedback of gamers and developers alike.

This time, Microsoft have been talking to industry experts from Digital Foundry, for the tech specifics, and to Gamasutra, to showcase what they’re doing for developers. This way, the industry experts can ask the questions the fans want to know and tell the story as they see it.

This is a marked improvement from what can only be described as a fumbled Xbox One announcement.

This could be the start of something very different

I’m really excited about what “Project Scorpio” has to offer and I’m likely to get one at some point in 2018.

I have a feeling that there’s more to “Project Scorpio” than just a hardware refresh and I can’t help but wonder if we’ll see changes to the way the games are delivered too.

If the Xbox One platform is going to be around for a long time, why bother creating a new game every time? There’s no reason why a games franchise like Forza or Halo couldn’t be delivered as a service with constantly updated content and graphics.

We’ll hear more about “Project Scorpio” at E3 in June. This will likely include the final name and design.

Console Tabs Coming to Windows?

I was overjoyed when I first heard that Microsoft would be improving the console in Windows 10 (yeah, really) and we’ve had a steady amount of updates since then – including a load of improvements in the Creators Update.

All the updates we’ve had so far have been great and all… but I’ve been begging to have tab support in the console. And yes I voted for it on User Voice!

This morning I saw a couple of interesting posts from Windows Central and MSPoweruser with reports of a “Tabbed Shell” interface coming to Windows 10 in a future update.

Obviously my nerdy mind immediately went to the command line. (See above for my mock-up of what it might look like.)

It suddenly becomes super-obvious that having tab support at the OS level makes so much sense. From an engineering point of view there is no point in having multiple teams working on their own implementation of tabs in Windows.

When something crashes you don’t want one tab to take out all of the others. With support built directly into the shell you can ensure stability and consistency across all applications.

Hopefully we’ll find out more at the Build conference next month… and hopefully I’ll be using PowerShell in a tabbed interface in the not too distant future!

Note: yes I know there apps which provide tabbed consoles in Windows. I’m really interested by Hyper, but it doesn’t feel ready for every day use yet.

Mindfulness Meditation

Like other millennials I spent a lot of time on the internet when I was growing up. I firmly placed my attention on the exciting world of operating systems and software applications. I learned how to write my own software and build my own networks of computers.

But there’s more to life than just one aspect, and so a decade ago I started to broaden my horizons from the scope of technology to find other subjects that interested me.

I quickly found that I had just as much interest in behavioural science, psychology, and self improvement.

A little bit of reading about meditation will give you the impression that it’s something worth doing – anyone who’s interested in self-improvement has probably come across a book about it and have been amazed by the seemingly incredible benefits in lists like this:

  • Reduced stress and anxiety
  • More energy and improved sleep
  • Better cognitive and emotional functions
  • Health benefits and more…

I had to find out more.

I’ve tried a number of forms of meditation over the years, but the one that has stuck with me the most is mindfulness mediation.

There isn’t much you need in order to get started:

  • About 15 to 20 minutes of free time
  • A quiet place without too many distractions
  • A timer or stopwatch
  • No expectations

Sit down in your quiet place in a comfortable position, set your timer for 15 minutes, and close your eyes.

All you need to do is be aware of the breath as it comes in and out of your nose. Breath slowly and concentrate on how the air feels as it hits your nostrils. Breathe deeply, but don’t try to control your breath – just experience it as it happens.

As you sit there you’ll notice that many thoughts arise to try and take your attention away from your breath.

The mind wanders, and this is what minds do, it’s not a problem or a mistake. The awareness of this fact is what the practice is all about. Over time you’ll learn the patterns of thoughts which try to take your attention away. Don’t fight it, but let it pass – and guide your attention back to your breath.

Your mind will wander like this many times, and that’s okay – just keep guide your attention back to your breath as many times as is necessary.

When your timer goes off, slowly open your eyes and you’ll be ready to go about your day.

This is the practice of mindfulness meditation in its most simplest form. Sounds easy? Not really…

Many people (myself included) are frustrated and uncomfortable at first. It’s a very strange sensation to realise that your mind jumps around like it does.

Equally, people often have expectations that all those benefits listed earlier must happen right away. This is simply not how it works. It takes time.

Mindfulness isn’t something that just happens – it’s something you need to work on. That’s why it’s called a practice.

After a while you’ll find that it does make a difference, and you’ll be mindful of this mind wandering in other aspects of your life.

I have found this to be especially noticeable when reading, writing, and working on many of the tasks required in software development. The pull is real, and can be very distracting if you let it. My generation has been accused many times of having a short attention span.

I suggest Mindfulness: A practical guide to finding peace in a frantic world (2011) as a really good foundation for bringing mindfulness into your every day life.