The Robot Dream Doesn't Clock Out — Brian's RDK Challenge Story

Picture this: A compact rover with a three-layer acrylic chassis rolls through a room on its own. Its camera catches a glimpse of a person walking by, and onboard, an ultralytics YOLO detector instantly fires back: person: 0.94. But that's just the first gate. A second, more discerning module takes over — comparing the operator's reference photo with the candidate, feature by feature. Blue jacket. Check. Bomber style. Check. Short black hair. Check. All three align. A moment later, a precise coordinate locks onto the SLAM map, flashing live on the web console.

That's VLM Patrol Robot — Brian's entry to the RDK Challenge Summer.

Meet Brian

Brian's journey started with wheeled robots in Malaysia. Today, he's at a robotics company in Abu Dhabi, working on humanoids. Wheel-legged-humanoid — he calls it "the dream path for anyone who builds robots." Ask him how far he'll go, and he answers with an engineer's quiet restraint: “one step at a time, we'll see.”

His new team is a global crew — colleagues from Africa, the US, the UK, the Netherlands — all drawn together by one thing, or rather, one love: making robots.

Why join a hackathon on top of a full-time job? His answer comes straight from the dreamer in him: "At work, you build what the client wants — you're boxed in by their scenario. A challenge is an open field. You get to show your imagination, build what you want. It's your Robot Dream, realized."

How Brian Found RDK

Brian didn't stumble upon RDK by accident. He went looking for it.

He had been using Raspberry Pi for AI projects, but over time the compute and ecosystem started feeling like a ceiling. So he did what good engineers do: he researched. Then one day he saw the RDK Challenge Summer announcement on LinkedIn -- and the timing couldn't have been more perfect. His discovery of the RDK X5 board and the competition launch landed almost exactly on top of each other.

Later, he shared his competition demo with colleagues in Abu Dhabi. Most of them had never heard of RDK. But after watching the video, they were hooked — asking detailed questions about what the board could do.

VLM Patrol Robot: A Self-Patrolling Robot

In one sentence: VLM Patrol Robot is a self-patrolling robot. It navigates autonomously on a SLAM map. Its onboard Ultralytics YOLO11n keeps a constant watch for pedestrians. When a candidate appears, a visual language model running right on the board — InternVL2.5-1B — compares the person against a reference photo, checking clothing color, garment type, and hairstyle. When all three features match, the target's coordinate is frozen on the map and surfaced in real time through a web console.

At the system level, this is a remarkably complete build:

• Hardware: A three-layer acrylic chassis rides on four mecanum wheels for omnidirectional movement. A LeTMC-520 RGB-D camera handles perception. Four wheel encoders plus an MPU-9250 9-axis IMU feed odometry data. There's even a small OLED inside the body, displaying runtime status and IP address.
• Perception & Navigation: The RGB-D depth stream is converted to laser scan data via depthimage_to_laserscan for gmapping. On the move, onboard AMCL handles localization. An EKF node from robot_localization fuses wheel odometry and IMU at 30 Hz, keeping position estimates rock-steady.
• Two-Stage AI Pipeline: ultralytics YOLO11n — an int8 model running on the BPU at 640×640 — handles continuous detection. InternVL2.5-1B steps in on demand for feature matching: the vision encoder runs on the BPU in int16, while the language head (Qwen2.5-0.5B) runs on the CPU. The two stages never block each other. ultralytics YOLO's detection feed stays live; the VLM match happens quietly in the background.
• Motion Control & Safety: Motion control lives in firmware on an RP2040 (Pico), talking to the main controller over micro-ROS. It enforces a 20 rad/s speed cap and a 0.5-second cmd_vel watchdog — if communication drops, the brakes slam instantly. On the hardware side, an RF remote emergency stop sits in series with the motor driver's power rail. Press it, and the power is physically cut. No software involved. No software to fail.

One more thing worth calling out: this system matches surface-level features only. It does not perform facial recognition. Brian drew that line deliberately and documented it. A submission that clearly states what it won't do often earns more trust than one that only boasts about what it can.

The Part That Almost Broke It

Not everything went smoothly. The hardest moment came when Brian tried to run YOLO detection and the VLM at the same time on the RDK X5. The board pushed to about 90% CPU and 70°C, and things started feeling unstable.

His fix was a mix of smart offloading and disciplined gating: run ultralytics YOLO and the VLM's vision encoder on the BPU, trigger the VLM only when ultralytics YOLO actually detects a candidate, and keep the heavy lifting off the main CPU wherever possible. That last part — offloading motion control to the Pico, so driving stayed smooth. Because a VLM comparison can take a long time, he also saved the robot's location at the moment of detection instead of after the match finishes.

What he's most proud of about this project is getting a Vision-Language Model to run fully on-board a low-cost RDK X5, describing people in plain language and matching them to a reference photo alongside live YOLO detection, with no cloud connection. Even with the whole stack running, teleop stays smooth and matches still land at the right spot on the map.

Three Words That Keep Him Going

At the end, Brian gave three words to describe himself: Curiosity. Perseverance. Imagination.

Curiosity — that thing where you take something apart just to see how it ticks. Not because it's broken. Just because you want to know. One minute you're wondering "what if?" and the next thing you know it's Sunday night, you've got a half-built prototype on your desk, and your browser has seventeen datasheet tabs open. Happens to the best of us.

Perseverance — the art of not giving up when everything is being difficult. The EKF drifts. The model throws a tantrum. The chassis wobbles for the fifth time. Perseverance is just... staying with it. Not because you're some genius who never struggles, but because you're the kind of person who'd rather figure it out than walk away. You've been there. You know the feeling when it finally works.

Imagination — seeing something that isn't there yet. Looking at a pile of parts and already picturing the finished thing. It's what lets you look past "this is how people usually do it" and ask "but what if I tried it my way?" Imagination is where every Robot Dream starts.

Here's the thing — you don't need to show up with all three fully charged. Maybe you're overflowing with curiosity, but your perseverance is still warming up. Maybe your imagination is off the charts, but you're still building your grit. That's fine. A challenge like this isn't a test you pass or fail. It's a gym. It builds what you've got and shows you where you can grow. Show up with whatever you've got. The rest comes along the way.

Next Dream. See You in Winter.

Registration for the RDK Challenge Winter (October 1 – December 15) is now open. Maybe the next story in this series will be yours.

If you’re still on the fence about joining, we hope what Brian shared finally gives you a little more courage to take the leap.

"Don't be afraid of failure. Keep trying, and you'll get there."

"When you want to do something, don't overthink it. Just execute."

"Your first step can be slow. But you have to take it."