Plenty of cruisers leave the passport in the cabin safe on port days, and Royal Caribbean's shore-excursion advice asks for a SeaPass card and photo ID, not a passport. The State Department's guidance points the other way: always carry your passport in case of an emergency. If you're hurt ashore and can't be discharged before the ship sails, it may leave without you, and you'd need a passport book to fly home. A passport locked in the cabin safe sails with the ship. If you lose it abroad, report it right away to the nearest U.S. embassy or consulate.
The one exception is a port where the ship holds passports for immigration, as Holland America says it may. Then follow the line's instructions.