Docs for the SAP repserver cover:

• Upgrading the repserver by overwriting an existing directory
• Upgrading the repserver by installing to a new directory
• Upgrading the whole repserver at once by calling the repserver executable with the “-upgr” option (aka the “one step” method)
• Upgrading the repserver using rs_init to upgrade the RSSD and all the databases connected to the repserver, but one at a time.

Unfortunately, the documentation pages covering this (within the Replication Server Configuration Guide for UNIX) overlap and jump back and forth in describing the various scenarios making them difficult to follow.

As a service to people who might  someday try this, here are my notes for doing a repserver in-place (ie. overwrite the files in the existing repserver directory) upgrade using the “one step” -upgr option (also called the “repserver upgrade utility” and the “seamless upgrade”):

• Replication Server Configuration Guide for UNIX 16.0 SP03 PL11
  • Upgrade or Downgrade SAP Replication Server
    • Preparing to Upgrade –  Mostly concerns upgrading from older versions of the repserver (eg. 12.6).   Need to make sure subscriptions are valid.
    • Upgrading the Replication Server
      • Stopping Replication Agent and Quiescing Replication Server (also see Upgrading Using Existing Directories, which is missing the quiesce step):

• • • • • Run “admin who” to get a list of all ASE servers accessed by the repserver.  Save this output.
        • Do ‘suspend log transfer from all’
        • The docs say stop the RSSD db rep_agent, but that only applies to repservers with routes
        • Run ‘admin quiesce_check’, wait for the repserver to be quiesced
        • It doesn’t actually say it here, but shutting down the old version repserver is implied.  Shutdown the repserver at this point.
        • Backup the RSSD
        • Backup the pre-upgrade repserver/<name> directory tree
      • Upgrading RSSD or ERSSD and User Databases with repserver ([sic] should say “using the one step upgrade utility”) – Says, before using the repserver one step upgrade option (ie, running the repserver executable with the -upgr option) you have to:
        • In the RSSD server grant sa_role to the primary user (get the primary user from the RSSD_primary_user line in the repserver .cfg file in the install directory):
          sp_role ‘grant’, sa_role, <primary_user>
        • In each ASE server accessed by the repserver, grant sa_role to the maintenance user
        • (get the maintenance user from the RSSD_maint_user line in the repserver .cfg file in the install directory)
          sp_role ‘grant’, sa_role, <maintenance_user>
        • Extract the SAP software distribution download into a directory to prepare to install the new repserver executable files
        • Run ./setup.bin to install the new repserver software into the existing repserver/<name> directory tree.  This will save old SYBASE.*sh files and create new ones.  It might also modify the .cfg file in the install directory
        • Log out and then log back in to the host, then source the new repserver/<name>SYBASE.csh file
        • Run the actual “one step” upgrade command (normal repserver executable call from the “RUN” file with -upgr added).  You might need to include a call to SYBASE.sh.  For example:

          #!/bin/sh
          source /home/sybase/repserver/myrepserver/SYBASE.sh
          /home/sybase/repserver/myrepserver/REP-16_0/bin/repserver -upgr -SMYREPSERVER 
          -C/home/sybase/repserver/myrepserver/REP-16_0/install/MYREPSERVER.cfg 
          -E/home/sybase/repserver/myrepserver/REP-16_0/install/errorlog_MYREPSERVER 
          -I/home/sybase/repserver/myrepserver/interfaces

        • This will upgrade the “Replication Server RSSD, and user databases in one step“ (does it do anything to the stable queue devices? Probably only for major releases)
        • When I ran a minor upgrade from 16.0 SP03 PL05 to PL11, the repserver -upgr process only displayed “RSSD has been updated successfully” and nothing else.  There were no messages about upgrading user databases. There was “no upgrade complete” message at the end.
        • After the repserver -upgr process completes it will exit (as opposed to continuing to run to do replication)
        • If there were any ASE upgrade errors in the -upgr procedure, fix things manually with rs_init.   See Fixing a Failed Replication Server Upgrade
        • Assuming the upgrade was successful, in the RSSD server revoke sa_role from the primary user
        • In all ASE servers connected to the repserver, revoke sa_role from the maintenance user
        • Then start the repserver normally.
        • Note, the post-upgrade site version (sysadmin site_version) doesn’t always match the actual software version (see docs).
        • Run “admin version,’connection’;” check if all databases connected to the repserver have been upgraded (“status” for each DSI should be “Database has been upgraded“)
        • Resume log transfer to resume replication.
        • Note, for newer versions of repserver 16.0, smp_enable has to be enabled.  See KBA 3045769.  smp_enable means use native OS threading, but “smp” is a misnomer here.  smp_enable has nothing to do with CPU processing working across multiple cores.  It really should be called “use native threads”.  It lets the repserver use real OS threads as opposed to a single OS process that emulates threading internally.