Retinitis pigmentosa takes peripheral vision and night vision first. Long before central vision goes, a badly lit street is a problem, and so is a corridor, a pub, a car park in December.
Phones are now fast enough to do something about that in real time. AuraPath takes the camera feed, brightens it and picks out edges, and puts the result back in front of you through a headset that costs a few pounds.
What exists
A working Unity prototype, running off a test video rather than a live camera. That is a real limitation and it is the next thing to fix.
The parts that work:
- Night-sight processing as a compute shader, brightening a dark feed without the noise that usually comes with it
- Edge detection, so the outline of a step or a doorframe survives when the detail does not
- Stereo rendering with barrel distortion for Google Cardboard, and a mono path intended for optical AR glasses
- A latency budget measured every frame, because a lag between moving your head and the picture moving is not a minor annoyance, it is nausea
The shape of it
Four layers, each behind an interface, so a piece can be swapped without touching the rest.
ICameraProvider -> IImageProcessor -> IRenderer
Today that is a video file, a night-sight shader, and a Cardboard stereo view. Live Android camera capture is written as a stub and is the next real piece of work. Optical glasses, the XReal Aura in particular, are the target after that.
Why a cheap headset
Dedicated low-vision headsets exist and cost thousands. A phone plus a Cardboard viewer costs a few pounds and most people already own the expensive half.
That is worth being careful about rather than pleased about. Plenty of projects have made this argument and shipped something unusable. Whether the processing is genuinely good enough, and whether anyone can bear to wear it for more than five minutes, are open questions and I do not know the answers yet.
This is what I am working on
Of everything here, this is the one getting attention at the moment. It is also the roughest, which is why the code is not public yet: opening a repository that does not yet do the thing it claims to do wastes the time of anyone who looks.
It goes public once live camera capture works and the whole loop runs on a phone rather than on a video file. At that point there is a lot to do and I would welcome the help.
Worth knowing in advance, if this sounds like your sort of thing:
- Unity 6, C#, URP, targeting Android
- Live camera capture is the piece being built now
- Real hardware testing is a gap I cannot close alone. I have a phone and a Cardboard viewer. I do not have an XReal Aura
- Comfort and latency is the difference between a demo and something somebody would wear, and it is where this will be won or lost
If you want to be told when it opens, say so and I will remember. It will be Apache 2.0, and the contributor terms apply.