What is Microsoft’s Holoportation project?
Holoportation is a futuristic mixed reality technology that’s being developed by a bunch of people at Microsoft. It uses three-dimensional mapping of objects of the real world to create 3D reconstructions which can be compressed and transmitted to a remote location via the internet. At the destination, these models are projected in real time and a person wearing Microsoft’s HoloLens is able to see, hear and interact with the objects from the source as if they’re together with him in the same room. A demonstration of Holoportation is shown in the video below: Holoportation is one of the first steps of fundamentally changing the way people communicate in the future. Now that we know what this technology is, let’s analyze it from as many viewpoints as possible. What’s a better analysis method than SWOT that organizations use for themselves and their products. SWOT stands for Strengths, Weaknesses, Opportunities, and Threats. I will deal with each one individually. But, before we get on to SWOT though, as required, I have written about my study and analysis of holoportation with respect to tracking, navigation and interaction and collaboration that we studied in previous classes of this course. Note: During my research for this week’s homework (and much later after I did the SWOT analysis), I came across news articles writing about Holoportation but more importantly other similar technologies that do exist, namely ARHT Media’s Humagrams (ARHT: Augmented Reality Holographic Technology) and Hologram USA which are seemingly far more advanced than what Holoportation is. This definitely changed my perception of Holoportation as I started disparaging what the creators at Microsoft did in contrast to my earlier amazed view of Holoportation. It felt like Microsoft is just the new kid in the block, experimenting and learning this new technology, whereas the companies that I’ve mentioned above have gone much further than still trying to reduce video and audio lag, producing sharper holographs which look a lot more realistic and getting the sync between the audio and video right. My first edition of this homework was more positive than negative but later I became even more critical and tried to bring out many negatives. Eventually, what I realized is that, it is not exactly right to make a direct comparison between holoportation and the work of the other two companies because in at least some ways they differ fundamentally, one of them being their target customers. The below analysis, among other things, is with respect to collaboration, tracking, navigation and interaction and how holoportation performs in comparison to ARHT and Hologram USA’s work. Tracking: The HoloLens used in Holoportation uses Inside-Out tracking rather than Outside-In tracking (used by VIVE and Oculus Rift). In inside-out positional tracking, the cameras or sensors are located on the device being tracked (e.g. HMD) while in outside-out the sensors are placed in a stationary location. However, to create 3D models for holoportation, Microsoft is forced to rely on 3D capture cameras placed at a height so eventually for holoportation, the tracking is Outside-In. Tracking in holoportation is limited to the spatial range of the cameras. If you go out of this range, you will no longer be tracked. The minimum number of cameras required is two, however, the more the cameras, the better the tracking and the better is the quality of 3D models produced. The 3D models are accurate enough to make them look like real people (after applying textures appropriately) rather than traditional animated avatars. Unlike avatars, the cameras are sensitive enough to capture almost every major movement of the real person. But there’s a bummer! The 3D reconstructions do not have sharp edges. The edges are either jagged in some places, blurred in some places or even portions of the avatar are found to be missing near the edges.
In the image above, notice the model’s left leg shoe. It’s so bad that it seems like it’s been eaten by rats! You can also see the blurring on the hand among other faults. Apparently, they haven’t given enough attention to the level of detail. Now, one would forgive Microsoft by thinking that they’re just developing something that’s in its nascent stage. But companies by ARHT media and Hologram USA have been producing near accurate and sharp 3D holographs since a long time now. This might be because Microsoft may be using either lossy compression or a poor capture technology.
One stark observation is the lag between the motion of the demonstrator, Shahram, and his corresponding virtual 3D model on the tv screen behind which implies that the latency of this system is quite considerable. But weirdly, when he puts on the HoloLens to see his own holograph, the holograph appears to have a much lower latency. Is Microsoft tricking the viewer or is it that the models are first generated, reconstructed for the HoloLens, and then displayed on the tv screen at a much later time such that the lag becomes more prominent? Who knows. The video snippet below shows this latency:
Navigation and Interaction: Holoportation mainly relies on a Fish Tank VR setup because you have to be within a certain area in order to be detected by the 3D capture cameras. Plus, once your 3D model is generated your movements should be restricted to that area for tracking so navigation is limited and defined by the camera range. It would be possible to increase the size of this area but then the cameras would be too far from the objects and the holographs would deteriorate in quality. In that case more cameras would be required which would increase costs. The user though, doesn’t need any controllers to move within that confined space. He is free to use his legs to walk around the space. Accordingly, the participant on the other end would see the user moving or making gestures. For augmented reality, the HoloLens reads the user’s gestures (like the ‘air tap’ and ‘bloom’) and outputs the expected action. The same gestures hold for holoportation. Gestures make it easier for the user to interact with the augmented reality environment rather than having to press buttons because buttons take time to adapt to but gestures, when selected well, are intuitive and easy to get used to. One feature that sets Microsoft’s holoportation apart from AHRT and Hologram USA is the ability that it gives users to interact in some ways with the holographic images projected through the HoloLens in their space. Users can record sessions that they had with a holoported participant and play them at any later time. The user can scale these replays to any size they like using gestures. This is shown in the image below: Now, I’d like to talk about what distinguishes Holoportation from ARHT and Hologram USA. The factor that I’m talking about is target customers. ARHT and Hologram USA have technologies that are mainly meant for companies and large-scale organizations who have enough capital to advertise or showcase their products using holographic tech. On the other hand, Microsoft’s sincere goal is to bring this experience to the home of a common man at an affordable price. From the videos I’ve watched of ARHT and the other, I’ve found that their projects are made either for films, tv shows, live on-stage performances, brand advertisement and other such things which are not in the scope of a common man. So, Microsoft’s wants to tap the middle-class consumer market, a large one, who would buy their product for communication and maybe recreation, and that’s what holoportation is for. The productions of ARHT and Hologram USA are mainly for people to watch from a distance than to interact with the holographs. But again, you don’t need a headgear like HoloLens to see ARHT’s humagrams and Hologram USA’s 3D holographs. Collaboration: Microsoft’s Holoportation does include collaboration in some sense because people in geographically different locations are able to view each other in their own spaces. But since this is augmented reality, collaboration here is much different than in virtual reality. People have awareness only of their own space and do not share an environment with other participants. They can talk to each other, hear each other but they can’t exchange virtual objects (Or can they?). Each person can have his own viewpoint and control it in holoportation. The two main issues with collaboration that we saw were i) loss of level of hierarchies in case of avatars and ii) collisions. Holoportation gets rid of the former because the holographs are closer to the real person than an animated avatar, but collisions are unavoidable. In the gif below, you can see Shahram moving back so that he doesn’t come in the way of his daughter’s moving holograph. You could easily walk through a holograph. Since the environment is not shared, one can’t reference any objects out of the ranges. Objects within the range are holoported and can be referenced. You can direct the participant on the other end to interact with an inanimate object because you can see it since it is present within the range. But since you cannot see objects beyond that, if you’re directed to some object like that, you’d not know where to look. Moreover, if you want to hand that bouquet of flowers to your holoported guest, it’s as good as useless because your guest won’t be able to receive it. One thing to wait and watch for is how Microsoft handles audio when there are too many holoported people. Because if everyone talks at the same time and if you can hear all of them with the same volume then it will just be noise. Microsoft could consider the distance factor and assign volume levels based on the distance between objects. Maybe Microsoft still hasn’t reached that level to deal with it yet. Another point that I observed was that while demonstrating holoportation, they did not use large distances between participants. The participants were either in a nearby room (Shahram’s video) or in another building across the street (TED talk of Alex Kipman wherein he talks to a holoported Dr. Jeff Norris: https://www.youtube.com/watch?v=1cQbMP3I5Sk ) which means they’re taking the advantage of using a faster network and not taking risks of transmitting data over huge distances. Smart move Microsoft. But you eventually gotta get holoportation working well over large distances. Moving on to the SWOT analysis below, which is to summarize the above study of Holoportation plus some additional points. Strengths:
Weaknesses:
Opportunities:
Threats:
In the end, Microsoft has by far done a pretty good job in Holoportation. Application developers can make the technology even more exciting once they start developing apps built for exploiting holoportation with the HoloLens. However, it has a long way to go before they release it commercially.
1 Comment
|
|