Even though the release of Apple’s Reality Pro mixed-reality headset is probably still a few months away, the company’s top-secret concepts are already starting to circulate. The most recent leak reveals an odd vision for how you might be able to control things in Apple’s metaverse.

Apple is reportedly experimenting with the notion of utilizing an Apple Pencil as a kind of virtual reality (VR) controller, according to a newly awarded patent (number 2023/0042447 A1). The concept is to overlay augmented reality (AR) features with your hand holding the Apple Pencil, displaying it in the mixed-reality environment you view via the headset.

Apple claims that sensors or cameras, such as those built into mixed reality headsets, might record a picture of your hand holding an Apple Pencil. The patent goes on to describe how you might create “pencil-like markings” in the virtual environment with that. A virtual conference room where you can write on a whiteboard and see the Apple Pencil make the lines in real time is one example.

Hand-tracking and gesture control

It’s a fairly cool concept and gives us a glimpse of what Apple could allow you to do with your hands when wearing the Reality Pro headset. We’ve seen a number of Apple’s concepts thus far, some of which incorporate current Apple goods.

A prior approved patent, for instance, demonstrated how two Apple Watches might be used to capture hand motions in the metaverse. Another of Apple’s proposals showed tiny gadgets the size of thimbles attached to a user’s fingertips, which would record hand motions and translate them into mixed reality.

The advantage of the Apple Pencil concept is that since many Apple fans already own one, it might not be necessary for many others to make a separate purchase. That would make it more affordable than a setup where you had to buy a pair of VR thimbles on top of the already expensive headgear.

This does not imply that Apple will certainly follow this path. The fact that the information is based on a patent raises the possibility that it will never be implemented and is only a concept that Apple is exploring but will never implement.

Even so, it looks like a savvy use of an already-existing Apple product to support the mixed-reality environment that the Reality Pro headset will produce. Regardless, we should learn soon because Apple is allegedly planning to unveil the Reality Pro headset at a special spring event. Keep a sharp lookout.