In 2018, I watched my buddy Allan engage the Mad Max autopilot mode on his Tesla Model S, cuing the car to switch lanes aggressively on Interstate 95. While the experience as a human was unnerving, the car leveraged cameras, sensors and artificial intelligence to maneuver safely.

Months later, I rode on a Boston Whaler 330 Outrage fitted with Mercury Marine’s Advanced Pilot Assist and Raymarine’s DockSense systems. As we approached the boat’s slip, the preproduction system used cameras, AI and the outboard engines to maintain a 3-foot safety buffer.

At the 2022 Fort Lauderdale International Boat Show, I saw these ideas meld in Avikus’ prototype NeuBoat autonomous operations system. The boat, with a human-in-the-loop operator, navigated itself out of its slip, up a river and around a lake before reversing course and docking itself.

Ready or not, autonomous technology is coming. This is likely good news for novice boaters—and for boaters who hate docking—because some of the marine industry’s smartest minds have been combining sensors and AI to smooth out boating’s rough corners. One example is NeuBoat (neuron plus boat), which Avikus is developing in partnership with Raymarine.

While experts say the sensors and software already exist to enable fully autonomous docking and navigation, Avikus and Raymarine foresee a road map to autonomy that earns trust with boaters while buying time for engine manufacturers to integrate the technology, and for agencies and organizations to create regulations.

“We’re intentionally paralleling the automotive market,” says Jamie Cox, Raymarine’s senior global product manager. “But I think we will beat automotive.”

Others agree. Sangwon Shin, Avikus’ director of strategic planning and business development, says: “In our view, the boating environment is less complicated than the car environment. So, we expect a little bit faster adoption rate.”

For boaters who are ready to start now, Avikus and Raymarine are releasing NeuBoat Dock this year. The assisted-docking system includes at least six self-calibrating, 360-degree cameras; a Raymarine multifunction display; an Avikus object-recognition unit; camera control boxes; and Avikus’ AI to provide bird’s-eye views and distance guides. (Garmin’s Surround View camera system provides similar capabilities.)

NeuBoat Dock is a level-one autonomous navigation system, which means it serves as a virtual assistant to human operators who remain in control. Level-two systems provide partial driving automation but still require a human operator. Level-three systems have conditional driving automation, requiring some human oversight, while level four has zero expectations of driver involvement. Level five is full driving automation.

Avikus, which is a spin-off of HD Hyundai, began developing NeuBoat in 2019. The resulting level-three-plus black-box prototype, which I got aboard in 2022, used the global navigation satellite system and vector cartography to establish position. The local device didn’t require internet connectivity. Instead, it employed daylight cameras and lidar (light detection and ranging) sensors to detect objects, measure distances, and scan and map berths. It also used Avikus’ AI to detect and classify nearby objects and vessels, assist with route planning, and suggest navigable courses.

This latter information was presented as screen views showing vector cartography with recommended courses, head-up displays and live camera views with augmented-reality data tags.

While impressive, the prototype didn’t use radar or the automatic identification system, so its range of object detection was limited to lidar’s 400-foot-range capacity. This range worked at our 6-knot speed, giving us 39 seconds of reaction time, but it wouldn’t work at 25 knots, only allowing for nine seconds.

Enter Raymarine, which integrated its own radar technology with Avikus’ AI. This combination extended NeuBoat’s detection range from 400 feet to 1.5 nautical miles. Shin says Avikus plans to integrate radar, sonar and infrared cameras within five years.

While extra range is important for recreational users, it’s critical for letting Avikus develop autonomous systems on large ships. “We use the same technology and the same algorithms for commercial and recreational, but the hardware specs are different,” Shin says.

In addition to radar expertise, Raymarine has amassed experience using computer vision from its DockSense and ClearCruise AR products. The latter places augmented-reality tags atop a video feed. Computer vision is a branch of AI that lets computers recognize, categorize and identify objects and people in digital images or video feeds; as such, it is critical to autonomous operations.

Looking ahead, Shin says, commercial ships and recreational vessels will first use autonomous navigation with human-in-the-loop operators, followed by autonomous operations. This isn’t a hypothetical; in 2022, Avikus’ commercial version of NeuBoat autonomously guided an LNG tanker across an ocean with human-in-the-loop oversight.

“The technology is there today,” Cox says. “We need to make sure that people are ready to use the technology responsibly and that regulations are there.”

When asked what milestones need to be met for autonomous operations aboard recreational yachts, Cox and Shin made clear they aren’t talking about distant horizons. “None are 10 years out,” Cox says, adding that by mid-2024, Avikus and Raymarine expect to have achieved sensor fusion, where the system can combine data from the vessel’s AIS, cameras, GNSS, lidar and radar. “In two years, on the control side, boats will be docking and driving themselves.”

Shin agrees: “In five years, we’re expecting lots of the boating community to accept the possibility of autonomous navigation or partial assistance on their boat.”

Before this can happen, however, Cox and Shin point to two technical complexities: networking with autopilots and engines. As with radars, Raymarine has decades of experience manufacturing autopilots, so engine interfaces could prove to be the sticky wicket. “Engine manufacturers need to become more progressive,” Shin says. “They are the powerful guys.” Cox says the goal is to integrate NeuBoat with every major engine manufacturer.

Cox and Shin also point to a need for regulations to govern autonomous vessels. This is already happening; in 2022, the American Bureau of Shipping published a white paper that detailed 10 points—from maintaining propulsion to maintaining communications—intended to create a structure for autonomous-vessel design and operations. The US Coast Guard also published guidelines on testing remote- and autonomously controlled vessels.

Convincing experienced boaters that autonomous technology is the path forward could be a hard sell for some, but this is where Avikus and Raymarine plan to parallel the automotive world. Most contemporary cars have adaptive cruise control, making these types of assistance features feel familiar. Many boaters also own cars with an autopilot feature.

But driving to work is different than taking the boat out for a spin. Here, Cox says NeuBoat isn’t going to take away boating’s joys. Instead, the idea is to reduce stress. For example, Cox describes allowing the boat to navigate autonomously to the fishing grounds or home from a cruise.

Cox also says autopilots have served boaters for decades, and that autonomous navigation is an extension of this capability, combined with the ability to avoid collisions autonomously.

For newer boaters, autonomous technology is an easier proposition. “I’m a new boater, and I get nervous a lot,” Shin says. “We target new boaters. We want more people to enjoy boating.”

Then there is boating’s greatest equalizer. “People don’t like docking,” Cox says. “We’re never going to stop you from driving your boat, but it might be nice, if you’re coming into a dock and are getting stressed out, to switch it on.”

The wait won’t be long, either. While Avikus is paralleling the automotive sector, Cox and Shin expect NeuBoat technology to navigate and dock recreational vessels sooner than cars. “People will be surprised with how quickly we will get to market,” Cox says.

Having experienced Tesla’s Mad Max mode and Avikus’ level-three-plus sea trials,

I can say that far less adrenaline is involved watching a demonstration boat dock itself than when I pawed for a nonexistent passenger-side brake pedal in my buddy Allan’s Tesla.

Better Optics

While NeuBoat Dock uses six 360-degree cameras, they only work for daytime operations. The obvious move is to add thermal-imaging cameras, and Raymarine’s parent company, Teledyne, owns FLIR. Thermal-imaging cameras would add cost, but Cox says these sophisticated optical sensors could be included aboard higher-end NeuBoat installations.

The post AI-Assisted Piloting Is Coming appeared first on Yachting.

It’s one thing to experience computer-assisted docking, but it’s different to ride aboard a vessel that’s autonomously negotiating the nautical road. I learned this during an on-water demo of NeuBoat technology from Avikus at the recent Fort Lauderdale International Boat Show. Our human pilot guided the demo boat out of its slip and into the Stranahan River. Minutes later, his hands left the helm, not to return. While it initially felt strange to place so much trust in silicone and sensors, trepidation morphed into amazement as we transited under the Southeast 17th Street bridge and into Lake Mabel. We passed hundreds of millions of dollars’ worth of gleaming fiberglass, aluminum and steel waterlines, but NeuBoat plotted a safe course through all of it and then back to, and into, the slip.

Like it or not, artificial intelligence is here, and it will only become a bigger, more integrated part of our world in the future. Hyundai Heavy Industries, the world’s largest shipbuilder and the parent company of Avikus, is already using autonomous capabilities to navigate ships across oceans, albeit with human oversight. While the NeuBoat isn’t the only autonomous-vessel technology afloat, it’s the only solution created by the commercial-marine sector with the parallel intention of innovating for the recreational-marine market. Avikus plans to offer two AI-assisted products, which can each be spec’d with two levels of operational capability.

The first is NeuBoat Navigation, which should be released in the second half of this year. It is designed to help human operators who are directly controlling their vessels make more situationally aware decisions. The basic option delivers navigational assistance through augmented reality to help skippers make smart choices, while the more advanced option provides navigation assistance along with camera- and sensor-collected informational assistance while docking.

The second product is NeuBoat Navigation and Docking Control, which Avikus plans to release in 2024. It’s a step up. Its basic option will include AI-controlled route planning, navigation and collision avoidance, with humans providing oversight. Its advanced option adds AI-controlled docking.

While  NeuBoat’s operating system and AI are scalable, the two systems (and their options) have different hardware requirements. NeuBoat Navigation installations have a black-box AI recognition processor and NeuBoat’s graphical user interface (GUI), which can be displayed on a compatible multifunction display or an Android-based tablet or smartphone. For a sensor, there is a forward-looking daylight camera and a forward-looking  light-detection-and-ranging (lidar) sensor. If applicable, NeuBoat can incorporate cameras for thermal imaging. Additional cameras and sensors can be added.

NeuBoat Navigation and Docking Control-equipped yachts will employ the same AI-recognition processor and GUI, but will also have an autonomous-control processor and an engine-interface module. Steering is via the vessel’s networked autopilot. The system will also use five to 10 daylight cameras and at least two lidar sensors for 360-degree situational awareness.

Both systems require either two networked global navigation satellite system receivers or a  networked satellite compass to  determine the  vessel’s position, heading and  rate-of-turn  information. The cameras collect imagery that’s sent to the system’s AI-recognition processor, where it’s compared, in near real time, with a growing image database that includes at least 2.5 million images. The system then uses a form of AI called computer vision to sort objects into one of eight buckets: motorboats, vessels, sailing yachts, rowboats, channel markers, buoys, structures or “other.”

“We determine obstacles in the image input from the front [camera] in real time with AI based on the pre-learned image data,” says Lim Dohyeong, CEO of Avikus.

The video stream is also sent to a helm display, where it can be seen by human operators. For navigation, the system can apply augmented-reality-style information tags above camera-captured targets, advising on their range and target type.

As mentioned, both systems employ a lidar sensor or sensors to determine precise distances. NeuBoat Navigation and Docking Control, however, also uses lidar to create distance maps between the sensor and objects in its surrounding environment (for example, docks and pilings) during autonomous docking.

“It scans the surrounding environment in real time whenever it enters or leaves the port,” Dohyeong says. The system basically creates a map from scratch each time—even if the dock is one that the yacht’s owners often frequent—to account for dynamic variables.

In addition to video feeds, part of NeuBoat’s GUI includes electronic cartography and a chartplotter like page view. The information serves as a database that the system uses for auto-routing and autonomous navigation. (For my demo ride, the system used an official electronic navigation chart; however, Avikus plans to integrate with third-party  vector-cartography  products, including C-Map and  Navionics.) This presentation also allows a human operator to understand the yacht’s position quickly, relative to landmasses, channels, navigational marks and other vessels.

As with other self-learning, AI-based systems, the more time NeuBoat spends navigating and capturing video imagery, the better it should perform. NeuBoat will be supported by biannual software updates.

While NeuBoat products have yet to be released, Avikus is working with several marine-electronics manufacturers on projects that will allow NeuBoat to incorporate third-party equipment, including AIS and radar, for collision-avoidance work. As an example, Avikus has signed a memorandum of understanding with Raymarine to collaborate on integrating NeuBoat technology with Raymarine’s product portfolio and to explore the  future of autonomous recreational  vessels  overall.

As of this writing, Avikus plans to let owners purchase NeuBoat Navigation as OEM equipment aboard a new build or in the aftermarket during a refit. Dohyeong says NeuBoat Navigation can even be added as a DIY project. Owners interested in NeuBoat Navigation and Docking Control systems will have to wait until they’re offered aboard new builds, but Dohyeong says aftermarket upgrades could become available for that system too, as its auto-calibration technology advances.

While adoption rates and boaters’ willingness to entrust potentially consequential operations to artificial intelligence remain an open-ended question, our test boat successfully navigated and self-docked. The gleaming nearby waterlines, I’m happy to report, remained unaffected by our autonomous passage.

Heavy Metal

In addition to NeuBoat,  Avikus builds HiNAS 2.0 (that’s  Hyundai intelligent Navigation Assistant System) for commercial mariners. In 2022, HiNAS 2.0 helped navigate a carrier full of liquefied natural gas across an ocean with human oversight. The ship’s fuel  efficiency increased by about 7 percent. Greenhouse-gas emissions dropped by about 5 percent.

The post Hands Free: Avikus’s Autonomous Navigation System appeared first on Yachting.

Garmin is taking boaters’ situational awareness to new heights—literally. With its Surround View camera system, the company aims to mitigate virtually all problems that skippers typically have when docking and maneuvering in tight spaces.

Using six cameras located around the boat, Surround View provides a bird’s-eye video and 360-degree views from the helm. For now, Garmin says, regardless of the boat’s size, there will always be six cameras: two amidships (one each to port and starboard), two foredeck cameras (again, port and starboard), one camera at the tip of the bow, and one far aft at the stern. Surround View stitches together the videos from each camera to create a seamless, real-time feed.

Skippers can also access each camera individually, with zooming and panning capabilities. The in-hull camera integration doesn’t compromise a boat’s exterior look. If a collision occurs, a springlike mechanism allows for the cameras to push into the boat. Damaged? On-the-spot repairs can be made without taking the boat apart.

Surround View also comes with augmented-reality features. Captains can see a visual bumper surrounding their vessels on their multifunction displays; that bumper can be expanded as close or as far as needed. There are also distance markers and a reverse-camera view akin to those found in cars. A Garmin chart plotter is required for Surround View to be integrated at the helm.  

The post Garmin’s Surround View Enhances Situational Awareness appeared first on Yachting.

The first time I rode in a car with 360-degree cameras was also the first time I experienced an acceleration curve that pinned me to my seat. My friend enthusiastically showed off the self-driving capabilities of his Tesla Model S. Mad Max mode was memorable, but I was most fascinated by the car’s external cameras. It was fun to snoop on unwitting drivers, even if the car’s autopilot and self-parking modes were using these external eyes for more-purposeful reasons.

I can imagine that Garmin’s engineers are familiar with go-park-yourself buttons because the company’s Surround View brings 360-degree camera views to select new-build helms. For these boat owners, docking and attaining better situational awareness just got easier.

History poorly recorded the first time a recreational yachtsman docked his steed, but stress and shouting were likely involved. After all, there was no auxiliary power or technology to simplify a task that, even today, ends marriages. While Garmin’s Surround View system won’t automatically dock a yacht (yet), the system employs a six-camera array, a black-box processor, advanced image-stitching capabilities and a Garmin multifunction display to give the helmsman a 360-degree video view around the yacht. Surround View can be used with third-party docking technologies, and in time, Garmin plans to upgrade the system’s capabilities.

“Even entry-level cars have bird’s-eye, 360-degree parking systems that let you see all around the car,” says Dave Dunn, Garmin’s senior director of marine sales. Surround View “gives you that full 360-degree image around the vessel to help you dock, or when you’re going out of the harbor or through a lock or channel. You can see your blind spots.”

Surround View consists of six Garmin-built, color 1080p daylight video cameras that are daisy-chained together and connected to the processor, which is networked to a Garmin multifunction display running the company’s Surround View app. The camera array consists of a single forward-looking camera, dual cameras on each board (port and starboard) and a single aft-facing camera. The forward-looking camera is bow-mounted, the four side-mounted cameras are through-hull secured, and the 180-degree stern-facing camera is placed up high.

Surround View is currently only an OEM option (see sidebar), but it can work aboard any kind of yacht up to 80 feet length overall. In all cases, boatbuilders work closely with Garmin to ensure that the cameras are properly installed with minimal blind spots.

The system captures the feeds from each of its six cameras and stitches them together before sending this composite imagery to the MFD, where it’s shown as a real-time, 360-degree video view. Users can also view single-camera feeds, and can pan and zoom within the system’s single-camera feeds.

Surround View also has Garmin’s Visual Bumpers and distance-marker features. Operators can define the Visual Bumper widths, and the system provides a graphical reference—overlaid atop the video feed—depicting water and nonwater objects. Distance markers act like camera lines in automotive-style backup cameras and give operators a precise graphical reference for how far their yachts are from a dock or nonwater object. Visual Bumpers and distance markers can be viewed separately or concurrently.

“There is some color differentiation—that’s a big part of it—but there’s also some [artificial intelligence] and augmented reality built into it,” Dunn says.

This differentiation is important when docking, but Dunn says the system can also differentiate swimmers, paddleboarders and kayakers from brine. Distances, he says, are calculated based on camera imagery, but laser-based technology such as lidar could potentially be used in the future. Surround View is currently a visual-only system; updates could include other types of alarms.

Surround View is already being used with Volvo Penta’s Assisted Docking system (see Yachting, July 2021), a hybrid manual/automated system with a GPS-based dynamic positioning system antenna. “If you pair Surround View with Volvo’s Assisted Docking, that’s a pretty powerful one-two punch,” Dunn says. “It’s safe to assume that we’re going to work with other engine manufacturers to do something similar.”

It’s also safe to assume that other innovations will follow. For example, Tesla’s Sentry mode uses the car’s cameras to record possible suspicious activity. Given that many of Garmin’s multifunction displays have HDMI video outputs, the company could potentially do something similar. “There’s great demand for a DVR-type function,” Dunn says.

While this type of recording feature could help bolster onboard security, there’s also fun to be had. Catch-and-release fishing tournaments commonly require competing teams to video-document their adventures to ensure rules compliance. Anglers commonly  employ GoPro or similar cameras; Surround View could be smoother.

Given that Garmin acquired Navionics in 2017 and, in 2020, released a multiband GPS 24xd sensor that provides position accuracy to about 3.3 feet, it’s fair to assume that Garmin will eventually leverage these and other assets to further enhance Surround View’s capabilities.

While Surround View is highly innovative, it’s not without a few drawbacks. The OEM-only system currently can’t trigger audible alarms or make autonomous adjustments. Also, every boat is different. “The biggest challenge we see right now is on an outboard boat,” Dunn says. “You have to get that camera high enough to be able to minimize that blind spot.”

Depending on the design, radar arches and tuna towers can be great camera-mounting locations.

Drawbacks aside, Surround View presents itself as extremely interesting technology for increasing situational awareness when it matters most. There’s no Mad Max mode—that’s probably a good thing—but odds are solid that the system will help more than a few marriages stay afloat. And the system should only improve as over-the-horizon capabilities hove into reality.  

Attaining Awareness

Garmin’s Surround View is one of the coolest advances in situational awareness, but it’s only available as an OEM option. As of this writing, Surround View is available on new builds from Absolute Yachts, Azimut Yachts, Groupe Beneteau and Solace Boats. This list is expected to grow quickly and substantially.

The post Garmin’s Surround View Eases Docking Stress appeared first on Yachting.

Volvo Penta’s Assisted Docking System is a hybrid between manual and automated docking, allowing users to maneuver their boat precisely.

To operate, Assisted Docking requires Volvo Penta’s Inboard Performance System and its GPS-based Dynamic Positioning System antenna. It also leverages a vessel’s steering, transmission and engine-control data.

The operator provides heading and speed input via the IPS joystick, and Assisted Docking moves the boat in a straight line at the user’s dictated speed. The system accounts for wind and current by reading the yacht’s position and heading multiple times per second, and Volvo Penta’s Electronic Vessel Control system steadily adjusts steering angles and thrust to keep the vessel on course.

One of the challenges of creating Assisted Docking was building a system that combines automation and manual driving in a way that simultaneously “gives the driver a good feeling of being in control and being supported by the system,” says Anders Thorin, Volvo Penta’s electronics product manager.

Assisted Docking does not use external components or sensors, allowing Assisted Docking-equipped yachts to berth anywhere, anytime.

Take the next step: volvopenta.com

The post Volvo Penta’s Assisted Docking System appeared first on Yachting.