MARVL Maritime RobotX 2026

Test record

We test at three levels: an automated suite, simulated missions against the real firmware, and sessions with the hardware in the water and in the air. This page lists what has been run, its objective and its result.

Test levels

Hardware time is limited. A pool booking takes a morning, a boat session takes a day, and the aircraft needs a licensed pilot present. We find as many problems as possible in software and simulation first, and use field sessions for the tests that need water or air.

Automated suite, every commit

The suite runs in about a minute and covers the mission logic, the protocol and the course model.

Simulated mission, before a hardware session

The mission runs end to end against the real firmware over an emulated radio, with a pass or fail per stage.

Hardware sessions

Autonomy, safety systems and failsafes are tested on the water and in the air. Each session is recorded from start to finish, and the footage is used as readiness evidence.

Software suite

The suite has 154 tests. The last full run took 53 seconds and passed 153. The remaining test is the software in the loop round trip, which is opt in because it builds and boots firmware.

Automated tests by area, as of 21 September 2026.
AreaTestsWhat it proves
Readiness rehearsal24The 3.1 requirement list, the course dimensions and the rehearsal criteria match the handbook
Course model22Course layout, the three live feeds and the browser scene build correctly
Suspend and resume21Task 4 interrupts a task and the task resumes at the step it stopped on
RoboCommand protocol21Envelopes, topics, sequence numbers and the run start handshake, end to end
MAVLink adapter21Message parsing, mode tables per firmware and stream setup
State machines16Every legal and illegal transition in the three machines
Software in the loop manager11Firmware process management, parameters and run directories
HaLow link emulator8Latency, loss, bandwidth cap and outage behaviour
Orchestrator5Startup, dispatch, shutdown
Embedded broker4Retained messages and subscription semantics for offline rehearsal
Software in the loop round trip1Opt in: boot, filter convergence, arm, a 40 m guided transit and an emergency stop

Simulated missions

The validation scenario is our simulated competition run. It launches the three firmware builds, drives them with the same ground station that drives the hardware, and checks 25 stages from a cold start, through a Task 4 interruption, to an emergency stop.

Simulated mission sessions.
SessionObjectiveDurationResult
Validation scenario, software in the loop Cold start to emergency stop with a Task 4 interruption in the middle, against the real firmware about 5 min per run 24 of 25 on the run of 21 September 2026. The open item is a guided waypoint that timed out on the boat; see below
Fringe link soak The same mission at 8 percent packet loss with a two second fade every 25 seconds per run Health monitor escalates to an emergency stop on heartbeat loss, as designed. Led to trimming the telemetry streams
Headless scene check Build the course in a browser engine with no screen, fly the mission, verify message order and camera behaviour seconds Pass in the suite whenever Node is installed
Readiness rehearsals The four handbook 3.1 demonstrations flown on the specified courses and judged criterion by criterion about 30 s each All criteria pass in rehearsal. See proof of readiness
Communications rehearsal The run start sequence of 3.4.10 against RoboNation's test server 1 min 11 s Run started with three vehicles declared

Field and water sessions

Hardware sessions and their evidence.
SessionSystemObjectiveResultEvidence
Autonomous operations, August 2026 BlueBoat Fully autonomous run with the status light showing the mode throughout Pass, submitted for readiness Video, 21 s
Autonomous flights, August 2026 Quadrotor Takeoff above 3 m, the published pattern and a landing, flown in Auto Pass, three missions recorded Video, 1 min 25 s
Manual flights, August 2026 Quadrotor The same pattern flown by the pilot, with mode proof on the ground station Pass, three missions recorded Video, 2 min 15 s
Geofence and ceiling, August 2026 Quadrotor Command the aircraft past the fence and above the declared ceiling Held at both limits Video, 36 s
Link loss failsafe, August 2026 Quadrotor Switch the transmitter off in flight and observe the response Landed five seconds after the link dropped Video, 42 s
Pool buoyancy, August 2026 Quadrotor Ditch the aircraft and recover it Floats upright and is recoverable by swimmer Video, 7 s
Bench motor test Quadrotor Motor order and direction before first flight Pass Video
Communications rehearsal, 31 August 2026 Ground station Run the handbook start sequence against RoboNation's test server Run 1 started, declaration sequence verified Log and screencast
Pipeline survey in water BlueROV2 Depth hold survey along a pipeline mockup with the magnetic probe Scheduled —
Three vehicle integration All One run with all three vehicles autonomous and reporting together Scheduled —

What testing changed

These test results led to design changes.

Underwater vehicle position

ArduSub does not allow guided modes without a position estimate, and the vehicle has no position estimate underwater. Depth hold is now used as the autonomous mode for that vehicle, and the survey is flown on heading and time with visual reacquisition.

Telemetry load on the radio

The default telemetry set overloaded the emulated link under the fringe profile. The adapter now requests position at 4 Hz and status at 1 Hz, and requests again if telemetry stops. This is a few packets per second per vehicle.

Arming delay

The filter needs 20 to 40 seconds after boot before prearm passes, and the position prearm check is the usual failure on the boat. Arming now retries for 90 seconds instead of failing the run.

Guided waypoint timeout

One check in the validation run fails: a guided waypoint on the boat does not report arrival inside its window, usually after the vehicle has been through an interruption. The mission still completes, so the cause is the acceptance window and retry logic, not navigation. This is the next item being worked on.

Reuse of the 2024 detector

The trained buoy model still works for the colour classes at 0.945 mAP@0.5. The shape classes do not transfer, because the 2026 buoys carry light beacons instead of shapes.

Planned tests before November

  • Pipeline survey in water with the magnetic probe, including a repair hold on a mock light node.
  • Docking approach and the water jet against a built docking bay.
  • Delivery mechanism flights: pick a tin, fly a route, release on a circle.
  • One session with all three vehicles autonomous at the same time, reporting through the ground station to a local broker.
  • Validation scenarios for Tasks 2 and 3 alongside the existing Task 1 run.
  • Fix the guided waypoint timeout so the validation run passes 25 of 25.