MRTK, the Microsoft-driven cross-platform toolkit for developing Mixed Reality (MR), Augmented Reality (AR), and Virtual Reality (VR) applications, offers a comprehensive solution for spatial interactions, UI functionalities, and rapid prototyping across various devices.
For those exploring options beyond MRTK, alternatives include OpenXR, Windows Mixed Reality, CloudXR, Unity, Oculus, SteamVR, OpenVR, and Vulkan.
OpenXR
Drift into the dynamic realm of OpenXR, the open standard solution bridging the divide in AR/VR software development. An innovation of the Khronos Group consortium, this specification promises seamless integration across various platforms and devices, with the emphasis on high performance and reduced latency.
OpenXR Top Features
High-performance access to AR/VR, eliminating the need for porting or re-writing code.
Comprehensive APIs, such as XrSpace and XrInstance, facilitate seamless interaction with diverse VR/AR hardware.
Extension mechanism extends the runtime beyond core features, allowing for customized implementations.
Fostered by powerful tech allies, OpenXR endeavours to extend its capabilities with constant updates.
Wide compatibility with engines like Unreal and Unity and platforms like Microsoft HoloLens 2 further strengthens its standing in the development community.
Feature
Description
Extension Support
OpenXR supports advanced features like world-scale experiences, spatial anchors, hand/eye tracking for immersive XR experiences.
Open-Source
Maintained by the renowned Khronos Group, OpenXR emboldens the open-source paradigm in the AR/VR landscape.
Interoperability
Seamless interop with other SDKs and OpenXR for WebXR only reinforces its compatibility prowess.
OpenXR Limitations
OpenXR is not an engine itself but enables engines to write portable code for cross-platform use.
The future development of OpenXR heavily relies on the advancement and widespread adoption of core API extensions.
Despite the diverse device compatibility, initial development releases focus only on platforms like Varjo PC.
OpenXR: Open Source and Free
Pushing the boundaries of accessibility, OpenXR is open-source and free, alleviating developer’s financial burden and fostering innovation.
OpenXR Use Cases
Use case 1: Gaming
With initial game engine support from Unreal and Unity, and its use in major titles like Minecraft and Flight Simulator, the potential of OpenXR in enhancing gaming experiences is undeniable.
Use case 2: WebXR Development
OpenXR with its browser support for Google Chrome & Microsoft Edge, facilitates immersive, seamless WebXR experiences.
Use case 3: Next-gen Graphic Technologies
The extensive extension support and hand, gaze tracking capabilities make OpenXR an ideal choice for developers aiming to leverage next-gen graphic technologies in VR/AR applications.
Windows Mixed Reality
An evolution in human-computer interaction, Windows Mixed Reality steps towards the future of computing by blending the physical and digital worlds. Working as a gateway beyond screen-bound experiences, it harnesses advancements in computer vision, graphical processing, display technologies, and more, to unlock intuitive 3D interactions between humans, computers, and their environments.
Top Features of Windows Mixed Reality
Interactive Layers: Goes beyond displays to offer spatial mapping and anchors, hand-tracking, eye-tracking, and more.
Range of Experiences: Accommodates a wide variety of VR experiences due to its inside-out tracking technology.
Collaboration-friendly: Facilitates collaboration on 3D assets in MR spaces.
Compatibility: Works with normal laptops, older PCs, and headsets from various manufacturers like Acer, HP, Asus, Dell, Lenovo, and Samsung.
Feature
Description
Versatility
The ability to smoothly transition between augmented reality and virtual reality.
Accessibility
There is a minimal setup requirement, aiming for mass market adoption.
Pioneering
Introduced the term ‘mixed reality’ to the world back in 1994.
Limitations of Windows Mixed Reality
Limited by the device’s sensors and processing power in creating new computer perceptions of environments.
Requires specific hardware (Holographic devices or immersive VR devices) to deliver the mixed reality experience.
Use Cases of Windows Mixed Reality
Enhanced Consumer Experience
Mixed Reality takes user experiences to the next level with holographic representations of people, 3D models, and the real world, thereby redefining consumers’ interaction with data within their living spaces.
Business Integration
For businesses, Windows Mixed Reality can be a new platform for providing immersive 3D interactions, advancing the enterprise’s capabilities with practical applications in training, product development, and more.
Social Media Integration
Windows Mixed Reality allows users to interact with their social media feeds, friends, and more through mobile Augmented Reality offerings, a prominent example being Instagram AR filters.
CloudXR
Welcome to the dynamic world of NVIDIA CloudXR SDK, a GPU-accelerated VR/AR streaming platform that allows you to stream OpenVR-based apps from a remote server, effectively utilising network technologies like Ethernet, Wi-Fi, and cellular. It’s designed to transform your XR experience, bridging the gap between Microsoft Windows and Android devices.
CloudXR Top Features
Connectivity with a range of VR clients including HTC VIVE headset, HTC VIVE Focus 3, Oculus Quest 2, and AR client Samsung Galaxy S20 via a 5G network.
Easy integration with SteamVR facilitated by a server driver, which encodes rendered frames and system audio, before sending to the client for decoding and display.
Optimised network performance with low-latency AR, VR, XR streaming, facilitated by RTX powered servers.
Category
Details
Supported Devices
HTC VIVE, HTC VIVE Pro, Valve Index and AR devices based on ARCore and ARKit.
Software Enhancements
NVIDIA RTX GPUs and NVIDIA RTX Virtual Workstation (vWS) for high-fidelity streaming across various platforms.
Industry Standards
Includes current OpenVR and OpenXR industry standards.
CloudXR Limitations
As comprehensive a tool as CloudXR is, it holds certain drawbacks:
Dependent on strong network connectivity for optimal functionality.
Performance is constrained by the quality and capacity of the server.
The scope of device compatibility, currently with explicit focus on HTC VIVE, HTC VIVE Pro, and Valve Index, may limit user access.
CloudXR Pricing
The pricing details of CloudXR were not provided in the summary. It is recommended to visit the official NVIDIA CloudXR page for the most accurate and up-to-date information.
CloudXR Use Cases
Use Case 1: Gaming
With its strong graphics capability and low latency, CloudXR offers an immersive gaming experience by streaming high-fidelity VR/AR experiences directly from OpenVR applications.
Use Case 2: Virtual Collaboration
Facilitating high-quality remote meetings, CloudXR offers virtual reality users the ability to share and interact within 3D graphics and modelling sphere, deepening the levels of collaborative engagement.
Use Case 3: Industrial Design
Professionals in the field of industrial design can make the most of CloudXR’s real-time 3D model review capabilities, streamlining the product iteration process.
Unity
Immerse yourself in the world of Unity, a dynamic game development engine birthed in 2005, turning creative visions into reality. Dive into the realm of 3D and 2D games that transcend boundaries, launching across Android, iOS, and various operating systems.
Unity Top Features
Versatility in game creation from Augmented Reality to 3D simulations.
Rich asset store with diverse pre-designed textures and features.
Compatibility with various coding languages including Javascript and C#.
Active developer community fostering ongoing collaboration and system improvement.
Feature
Description
Unified platform
Unity’s cross-platform game development simplifies deployment for developers.
Up-to-date tech
Continually evolving technology enhances user-friendliness, compatibility, and visual experiences.
AR/VR Creation
Unity stimulates mobile games and AR/VR feature development.
Unity Downsides
Unpredictable pricing redesign exerting pressure on solo, indie, mobile developers.
Open mistrust from developers due to unannounced changes.
Frustrations over potential abuse of per-install fees.
Unity Pricing
Unity’s new proposed pricing model imposes a fee each time a game using Unity software is installed. Notably, developers will not be charged an install fee unless their game surpasses $200,000 in revenue and 200,000 installations. Furthermore, games bought in standard markets bear a higher fee than similar purchases in emerging markets.
Unity Use Cases
Use Case 1: Mobile Game Development
Unity offers cross-platform gaming potentials, being adaptable with numerous operating systems, thereby benefitting mobile game developers.
Use Case 2: Augmented Reality (AR) and Virtual Reality (VR) Games
Unity’s robust AR/VR toolkit creates compelling and immersive experiences, inspiring a generation of AR/VR developers.
Use Case 3: 3D Simulations
Developers in need of creating advanced 3D simulations can leverage Unity’s comprehensive tools, and rendering technology.
Oculus
Reviving the VR industry with innovation and accessibility, the Oculus Rift has helped usher in a new age of virtual reality. Merging novel technologies into a VR headset that offered a realistic experience at an attainable price, Oculus stands as a behemoth in the 3D Graphics & Modeling sphere.
Oculus Top Features
Immersive 90 degrees horizontal and 110 degrees vertical stereoscopic 3D view.
Field of View (FOV) more than 90 degrees horizontal, double the FOV of previous VR devices.
Advanced motion tracking using three-axis gyros, accelerometers, and magnetometers for absolute head orientation.
1080p LCD panel for significantly improved clarity and reduced screen door effect.
Feature
Benefit
Geometric pre-distortion
Enables a large stereoscopic field-of-view
Low-persistence of vision (Crystal Cove prototype)
Drastically reduces blurring and judder
Oculus Limitations
The Oculus Rift line, known for its potent combination of accessibility and advanced features, was discontinued in April 2021.
Despite a large and varied software library, the switch from Oculus Rift to Oculus Quest may not be seamless for all users.
Oculus Use Cases
Use case 1
With its detailed graphics and immersive experience, Oculus Rift is ideal for gaming. It presents an alternative world, transporting users beyond the limits of reality.
Use case 2
For professionals in architecture and 3D design, the world of Oculus can be a powerful tool to create, manipulate, and visualize structures in an immersive environment.
Use case 3
Education and training can rise to new heights with Oculus, whether it’s to explore the realms of science, history, or training for complex procedures and operations.
SteamVR
SteamVR, an integral component of the Steam ecosystem, offers immersive virtual reality experiences. Primarily intended for gaming, it serves as a launching pad for VR experiences while offering in-app purchases, overlay applications, and 3D collectibles among other features.
SteamVR Top Features
Room setup for defining play area allows an immersive 360-degree VR experience
Device management ensures smooth PSU firmware updates and pairing of new devices
Overlay applications can be created for added functionality
SteamVR Home, a launching pad for VR experiences, facilitates app launches and friend connections
SteamVR Collectibles grants 3D items to players based on game ownership and playtime
Feature
Description
Compatibility
Supports its own HMDs (like Vive) as well as other VR headsets (like Rift)
Accessories
Supports accessories like base stations and input devices like SteamVR Controllers for positional tracking
Hardware
Compatible with multiple headsets such as Valve Index, Oculus Rift&S, HTC Vive
SteamVR Limitations
In mid-2022, users reported compatibility issues with Meta Quest 3
Requires specific system requirements: OS (Windows 7 SP1, Windows 8.1, or later, Windows 10), Processor (Intel Core i5-4590/AMD FX 8350 or better), and Graphics (NVIDIA GeForce GTX 970, AMD Radeon R9 290 or better)
Requires physical accessories like base stations for proper VR experience
SteamVR Use Cases
Use case 1: Gaming
SteamVR proves particularly valuable for gamers, offering an immersive 360-degree gaming experience powered by room setup options, device management, and compatibility with multiple headsets.
Use case 2: VR App Development
For VR app developers, SteamVR presents opportunities for overlay application development, supported in-app purchases, and compatibility with multiple VR headsets.
Use case 3: 3D Graphics and Modelling
Professionals in 3D graphics and modelling can utilize SteamVR for realistic rendering of 3D environments and SteamVR Collectibles to create and distribute 3D items.
OpenVR
OpenVR, the reputable VR SDK and API brainchild of Valve, serves as a bridge between VR hardware and software, bringing multi-vendor functionalities to your fingertips.
OpenVR Top Features
Multi-vendor support: Unites diverse VR hardware such as HTC Vive, Oculus Rift and Windows MR
SteamVR integration: Serves as the default SDK for HTC Vive
Open-source: Enables diverse collaboration under the Open Source Virtual Reality project
Plugin and Unity Support: Offers integrated plugin and native support in Unity
Tracking Features: Comes equipped with IVRChaperone and IVRSystem::ResetSeatedZeroPose for precise tracking data
Feature
Description
Class Reference
Includes detailed constructors, fields, properties, methods backed by Microsoft’s MRTK documentation
Unity Versatility
Supports several Unity versions from MRTK2 Unity 2018 2.5.0-2.7.0 to MRTK2 Unity 2020 2.5.2-2.6.0
VR System Controls
Allows initialization and shutdown of the VR system through Init(EVRInitError, EVRApplicationType, String) and Shutdown()
OpenVR Limitations
Requires proficiency with VR SDKs and APIs
Limited support for older APIs in steamvr.h in the Steamworks SDK
Complex tracking prediction may pose learning curve
OpenVR Pricing
As an open-source project, OpenVR is your passport to a sea of VR possibilities free of charge.
OpenVR Use Cases
Use Case 1 – Game Development
Given its multi-vendor support and robust integration with SteamVR, OpenVR stands as the perfect toolkit for game developers…
Use Case 2 – Software Design
The open-source nature of OpenVR offers software designers an expansive playground to experiment, innovate, and design…
Use Case 3 – 3D Modeling
With support for multiple tracked objects, models, and textures, OpenVR is a potent resource for 3D modelers…
Vulkan
Vulkan is a standards-aligned, cross-platform graphics API that empowers developers to create high-performance applications on a wide range of devices. With a continuous cycle of evolution, it integrates innovative functionalities like hardware-accelerated video compression and provides granular control over various aspects of graphics programming.
Vulkan Top Features
Extension Support: Frequent addition of new extensions and capabilities.
Hardware Acceleration: Integrated support for H.264 and H.265 video compression and decompression.
Fine-Tuned Control: Full control over scheduling, synchronization, and memory allocation for optimization.
Ray Tracing Integration: Efficiently integrates ray tracing into the Vulkan framework.
Vulkan Memory Model: First graphics API with a formal memory model for optimal memory operations.
Feature
Description
Vulkan SC 1.0
Caters to safety-critical systems balancing graphics and compute requirements.
Open-source Contributions
Encourage improvements to Vulkan spec or contributions to various software/tools projects.
Hybrid Viewport Technology
Implements photorealistic look and feel in real time rendering.
Vulkan Limitations
Deployment timelines and device support can lack clarity for developers.
Requires a concrete understanding of graphic programming concepts.
Add-on extensions may result in platform fragmentation.
Vulkan Use Cases
Use case 1: Graphic Programming Beginners
Vulkan provides an ideal platform for individuals initiating their journey in graphics programming, thanks to its robust extensions and adaptive frameworks.
Use case 2: Real-time Rendering
Vulkan’s hybrid viewport technology offers real-time, photorealistic rendering, making it an ideal choice for developers working on graphic-intensive projects.
Use case 3: Safety-Critical Systems Development
With the Vulkan SC 1.0 specification, Vulkan aligns the needs of graphics and compute requirements, reinforcing its utility in the development of safety-critical systems.
Grant Sullivan
Content writer @ Aircada and self proclaimed board game strategist by day, AI developer by night.
