Nov 14, 2023 15:00:00

D-POINT is a stylus pen that realizes 6DoF, which captures the movement of the pen with a camera and draws digitally.

A stylus pen, a digital pointing device that comes with tablets, smartphones, etc., usually transmits input information to the device by touching or approaching the pen tip with the monitor. The newly introduced ' D-POINT ' allows you to input characters by using a camera to read the marker attached to the end of the pen.

GitHub - Jcparkyn/dpoint: Open-source digital stylus using camera tracking and inertial measurements

https://github.com/Jcparkyn/dpoint

You can see how D-POINT works in the video below. Click to play.

The internal structure of D-POINT is like this. There is XIAO's SoC '

nRF52840 ' at the end of the pen, a lithium-ion battery that can be recharged with USB Type-C inside, and a force sensor at the tip of the pen. The pen tip is plastic.

The end of the pen has 8 markers from another project for marker detection '

ArUco '. By reading this marker with a camera, it is possible to read the position, posture, and orientation of the pen.

As for motion processing, the camera first detects the ArUco marker, corrects its position, and estimates the pen's posture using

Perspective-n-Point . After composing, the position and direction of the pen tip is calculated and drawn on the screen.

D-POINT can also detect pen pressure, achieving 6DoF input with sub-millimeter accuracy. As long as you have a pen and a commercially available web camera, you can draw on any surface.

The code and design files for this project are open source.