Certificates

Certificates

CMS requires encrypted communications between various components and as a result, X.509 certificates are required for all CMS deployments. They help ensure that a service/server can be trusted by other devices/services.

There are three types of certificates to consider: self-signed, private certificate authority (CA)-signed, and public CA-signed.

  • Self-signed certificates are certificates that the device can generate itself. They are not recommended or supported for most clustered/scalable deployments. Only the all-in-one-box deployments support self-signed certificates, and even then, it is not recommended.
  • Public CA-signed certificates require registration with a public certificate authority and are a requirement when interfacing with devices on the Internet. When deployed correctly, they are automatically trusted by browsers and mobile apps that connect to CMS. The downside is that these certificates cost money and require the domain to be registered with an external domain registrar.
  • Private CA-signed certificates are similar to the public CA-signed certificate except the CA itself is controlled by the customer and therefore not automatically trusted by devices outside the administrative control of the company managing the CA. That said, trust can be achieved by installing the root certificate from the private CA onto the devices that need to trust the certificates signed by the CA. This is the type of CA we will use in this lab.

Although every service requires a certificate, creating individual certificates for each service can add unnecessary complexity. Fortunately, you can generate a certificate public and private key pair, then re-use it for multiple services. In this lab you will obtain a single certificate for Call Bridge, XMPP server, Web Bridge, and Web Admin. With some additional considerations taken into account in advance, this certificate will also be able to integrate with on-premise Microsoft Lync / Skype for Business.

Database clustering has some special certificate requirements described later and therefore each database cluster member requires both a database server and database client certificate. These are almost always signed by the same, private CA.

For all CA-signed certificates (both public and private), the general process is as follows:

  1. The pki csr command creates a new private key and a certificate signing request (CSR) file on CMS.
  2. Download the CSR file to the local PC via SFTP.
  3. Submit the CSR to the Certificate Authority to get a certificate file issued.
  4. Upload the certificate to CMS via SFTP.

NOTE: The entire process of downloading the certificate signing requests, having them signed, and then re-uploading the certificates to CMS will be performed on the Remote Desktop session labeled PC1.

Once uploaded to the CMS servers, the certificates are available for use. The certificates just need to be assigned to the components that require them. You will do this step as you go through and configure the individual services.

Create Certificate Signing Requests (CSR)

This section will walk you through generating a Certificate Signing Request, getting it signed, and installing the certificates. As mentioned earlier, you will be using a single certificate for Call Bridge, XMPP server, Web Bridge, and Web Admin. You will then generate separate certificates for the database.

The document Certificate Guidelines for Scalable and Resilient Deployments goes into detail on all the certificate requirements for a CMS deployment. We highly recommended you read through this document before you do a production deployment of CMS. This lab follows the guidelines from this document. These are the requirements for the combined non-database certificates:

  • When deploying a single CMS server with one or more stand-alone roles, the CN is set to the Fully Qualified Domain Name (FQDN) of the CMS server as specified in DNS, e.g cms1a.pod2.cms.lab. When used as part of a Call Bridge cluster where each server has multiple roles, as we do in this lab, especially when integrating with Microsoft Lync / Skype for Business, the CN is crucial, so we will set the CN to a DNS name we have set up for the CMS Call Bridge cluster, cms1.pod2.cms.lab.
  • The subjectAltName includes the FQDN of the CMS Call Bridge, for example cms1a.pod2.cms.lab. This is the DNS name for the specific CMS server, not the whole cluster
  • The subjectAltName includes the XMPP domain, which is the domain portion of the URI that users will use when logging into CMS, e.g. conf.pod2.cms.lab
  • The subjectAltName includes the a DNS name of either the server FQDN or of a DNS name that indicates the web bridge cluster. In our case, it's the same 3 servers, so we could use cms1.pod2.cms.lab, but but to distinguish it as the web bridge cluster of servers, we'll use join.pod2.cms.lab

The pki csr command has the following syntax:

pki csr key/cert_basename CN:value subjectAltName:value

The key/cert_basename is just the file name that will be used for the key and certificate files. Different file extensions will be appended to this base name depending on the type of file. It can be anything (as long as there aren't any special characters), but it's usually best to give it a name that reflects the server/service that you're generating the certificate for, like servername or servername-service. There are many other optional parameters: O: for Organization, L: for location, and so on. None of these are mandatory, especially for a private CA-signed certificate.

NOTE: Only use letters or numbers for the name. No periods, dashes or other characters! The error returned is not very intuitive (“Invalid CSR name. Must be alphanumeric”). Between the subjectAltName fields, you also need to take care not to have any spaces or other characters (“Invalid argument” errors). Only names in a comma-separated format.

Use these links to connect to your CMS servers:

Cisco Meeting Server Name Password
cms1a.pod2.cms.lab c1sco123

You will start with the combined certificate on cms1a:

cms1a> pki csr cms1a CN:cms1.pod2.cms.lab subjectAltName:cms1a.pod2.cms.lab,pod2.cms.lab,conf.pod2.cms.lab,join.pod2.cms.lab ................... ....................... Created key file cms1a.key and CSR cms1a.csr CSR file cms1a.csr ready for download via SFTP

This exact same procedure was already done on cms1b and cms1c. Only the server name in the subjectAltName field is different and, of course, the file name, to make it easier to keep track.

Before downloading the CSR via SFTP, follow the next set of steps to generate the CSRs for the database components.

Database Certificates

CMS uses a postgres database with a single master and multiple, fully-meshed replicas. There is only a single master database at a time (the “database server”). The remaining cluster members are replicas or “database clients”.

For redundancy to work, database clusters must consist of at least 3 servers but no more than 5, with a maximum round-trip time of 200ms between any cluster members. This limit is more restrictive than Call Bridge clustering, so it is often the limiting factor in geographically dispersed deployments.

The database role for CMS has some unique requirements. Unlike other roles, it requires a client and server certificate, where the client certificate has a specific CN field that is presented to the server.

For the database cluster, a dedicated server certificate and client certificate are required. These must be signed certificates, typically by an internal private CA. Because any of the database cluster members may become the master, the database server and client certificate pairs (containing the public and private keys) must be copied to all of the servers so they can assume the identity of a database client or server. Additionally, the CA’s root certificate must be loaded to ensure that the client and server certificates can be validated.

Create Database Server CSR

Like before, use the pki csr command to generate the CSR. Connect to cms1a and issue the command below.

Cisco Meeting Server Name Password
cms1a.pod2.cms.lab c1sco123
cms1a> pki csr dbclusterserver CN:cms1a.pod2.cms.lab subjectAltName:cms1b.pod2.cms.lab,cms1c.pod2.cms.lab ......................................... ......... Created key file dbclusterserver.key and CSR dbclusterserver.csr CSR file dbclusterserver.csr ready for download via SFTP

These steps have already been performed on cms1b and cms1c.

Create Database Client CSR

Next, create the database client certificate. This one is unique in that it requires setting CN:postgres. No other fields, such as the machine FQDN or other information, is required.

cms1a> pki csr dbclusterclient CN:postgres ................................. ................ Created key file dbclusterclient.key and CSR dbclusterclient.csr CSR file dbclusterclient.csr ready for download via SFTP

Again, these steps have already been performed on cms1b and cms1c.

Download All CSRs and Database Key Files

Now that you have created all the necessary certificate requests, you can download them.

You will need download the key files and the signing requests using the PC1 Remote Desktop session. The reason for this is that this machine is part of the Windows Domain, allowing you to easily use the command-line interface to sign the certificates. Depending on your Certificate Authority, the process for getting the certificates signed will be different. But in all cases, the files must be downloaded first and CSRs signed.

  1. Access the Remote Desktop session for PC1.
  2. Double-click the CMD icon on the Desktop in PC1. This will bring up a CMD window in the Desktop\Certificates directory. If necessary, you could navigate there directly with the cd %userprofile%\Desktop\Certificates command from PC1.
  3. Now you can SFTP from PC1 to cms1a and retrieve the following certificate requests:
    • The combined server certificate request, cms1a.csr
    • The database cluster server certificate request, dbclusterserver.csr
    • The database cluster client certificate request, dbclusterclient.csr

    Follow these steps to get the files from server cms1a to PC1 by pasting each command into the CMD window on PC1:
  4. NOTE: After entering the psftp command for a host, if you are asked if you trust the host, just enter y. To paste into the cmd window on the PC1 Remote Desktop, just right-click on the cmd window and select Paste.
    psftp admin@cms1a.pod2.cms.lab Using username "admin". Using keyboard-interactive authentication. Please enter password: c1sco123 Connected to cms1a.pod2.cms.lab. Remote working directory is / psftp> get cms1a.csr remote:/cms1a.csr => local:cms1a.csr psftp> get dbclusterserver.csr remote:/dbclusterserver.csr => local:dbclusterserver.csr psftp> get dbclusterclient.csr remote:/dbclusterclient.csr => local:dbclusterclient.csr psftp> exit

      OR to enter all commands at once, click below:
    psftp admin@cms1a.pod2.cms.lab c1sco123 get cms1a.csr get dbclusterserver.csr get dbclusterclient.csr exit
  5. You should now be able to see the following 3 files using the dir command on PC1:
    • cms1a.csr
    • dbclusterclient.csr
    • dbclusterserver.csr

Sign All CSRs

You now have 3 CSRs that need to be signed: cms1a.csr, dbclusterserver.csr, and dbclusterclient.csr. We have provided an internal Microsoft Certificate Authority to sign these requests. Follow the instructions below to upload the requests to the CA.

  1. Begin by accessing the Remote Desktop session for PC1.
  2. From the CMD window on PC1, issue the following:
    certreq -submit -username CMS\cert -p c1sco123 -policyserver "cmslab-ad.cms.lab\cms-CMSLAB-AD-CA" -config "cmslab-ad.cms.lab\cms-CMSLAB-AD-CA" -attrib "CertificateTemplate:ClientandServerAuthentication" cms1a.csr cms1a.cer certreq -submit -username CMS\cert -p c1sco123 -policyserver "cmslab-ad.cms.lab\cms-CMSLAB-AD-CA" -config "cmslab-ad.cms.lab\cms-CMSLAB-AD-CA" -attrib "CertificateTemplate:ClientandServerAuthentication" dbclusterclient.csr dbclusterclient.cer certreq -submit -username CMS\cert -p c1sco123 -policyserver "cmslab-ad.cms.lab\cms-CMSLAB-AD-CA" -config "cmslab-ad.cms.lab\cms-CMSLAB-AD-CA" -attrib "CertificateTemplate:ClientandServerAuthentication" dbclusterserver.csr dbclusterserver.cer

Now you should have 3 certificates in your Certificates folder: cms1a.cer, dbclusterserver.cer, and dbclusterclient.cer. You can check this with the dir *.cer command on PC1.

Download Root CA Certificate

Since all of the certificates are trusted by the same Certificate Authority, in many instances you will need to supply the CA's certificate as well. This is another file that can very easily be downloaded from the Certificate Authority that will be used to sign all of our certificate requests.

  1. Access the Remote Desktop session to PC1.
  2. From the CMD window on PC1, enter the command: WGET http://10.0.224.140/certificates/cmslab-root-ca.cer.

This will download the root CA file in base-64 encoding to your Certificates folder on PC1 and name the file cmslab-root-ca.cer.

Create a Certificate Bundle

Some services, such as the XMPP clustering server, require a trust bundle to identify all certificates of clients that it will accept, as a form of authentication. To create such a bundle, you must create a file that contains all three server certificates (from cms1a, cms1b, and cms1c) in a single file.

For this lab, the server certificates for cms1b and cms1c have already been already created for you. You just need to download them by following these steps:

  1. Access the Remote Desktop Session on PC1.
  2. From the CMD window on PC1 make sure you are still in the Certificates directory (Desktop\Certificates).
  3. Now enter the following commands into the CMD window on PC1 to SFTP to cms1b and get the cms1b.cer file:
    echo get cms1b.cer | psftp cms1b.pod2.cms.lab -l admin -pw c1sco123
  4. Now SFTP and download cms1c.cer from cms1c with this command:
    echo get cms1c.cer | psftp cms1c.pod2.cms.lab -l admin -pw c1sco123
  5. Now you can create the certificate bundle file (cms1abc.cer) by entering:
    copy cms1a.cer + cms1b.cer + cms1c.cer cms1abc.cer
  6. At this point the Certificates folder on the Desktop of PC1 should contain the following 7 certificates, which you can verify with the dir *.cer command:
    • cms1a.cer
    • cms1abc.cer
    • cms1b.cer
    • cms1c.cer
    • cmslab-root-ca.cer
    • dbclusterclient.cer
    • dbclusterserver.cer

Upload Certificates, Root CA Certificate, and Key Files

Finally, you need to upload the appropriate files back onto the CMS servers. Each server will get its own certificate, the certificate bundle containing all three server certificates, and the certificates for the database server and client including the private keys. Because the database CSRs were generated on cms1a, that server already has those private keys, so you do not have to re-upload them.

To upload the files back to the CMS servers use SFTP and the psftp client on PC1 as follows.

  1. Access the Remote Desktop session to PC1.
  2. Access the CMD window on PC1, which should still be in the Desktop\Certificates directory.
  3. Follow these steps to upload the files to cms1a from PC1 by entering the commands into the CMD window on PC1:

    psftp admin@cms1a.pod2.cms.lab Using username "admin". Using keyboard-interactive authentication. Please enter password: c1sco123 Connected to cms1a.pod2.cms.lab. Remote working directory is / psftp> put cms1a.cer local:/cms1a.cer => remote:cms1a.cer psftp> put dbclusterserver.cer local:/dbclusterserver.cer => remote:dbclusterserver.cer psftp> put dbclusterclient.cer local:/dbclusterclient.cer => remote:dbclusterclient.cer psftp> put cms1abc.cer local:/cms1abc.cer => remote:cms1abc.cer psftp> put cmslab-root-ca.cer local:/cmslab-root-ca.cer => remote:cmslab-root-ca.cer psftp> exit
      OR to enter all commands at once:
    psftp admin@cms1a.pod2.cms.lab c1sco123 put cms1a.cer put dbclusterserver.cer put dbclusterclient.cer put cms1abc.cer put cmslab-root-ca.cer exit

  4. Now upload the files to cms1b and cms1c. Since those CMS already have their server and root CA certificates, you only need to upload the trust bundle, cms1abc.cer. This is done with the following commands from the CMD window on PC1:
    psftp admin@cms1b.pod2.cms.lab c1sco123 put cms1abc.cer exit psftp admin@cms1c.pod2.cms.lab c1sco123 put cms1abc.cer exit

You won't need to use PC1 again until later in the lab when you will need to use it as a client for placing test calls.

Cisco Meeting Server Name Password
cms1a.pod2.cms.lab c1sco123

To make sure the certificates were uploaded correctly, you can ssh to CMS and use the pki list command which will give a list of the file names (The ones in bold are the the ones that will be required for proper operation. The .csr files are no longer needed.) Take a look at CMS1a:

cms1a> pki list User supplied certificates and keys: cms1a.key cms1a.csr dbclusterserver.key dbclusterserver.csr dbclusterclient.key dbclusterclient.csr cms1a.cer cms1abc.cer dbclusterclient.cer dbclusterserver.cer cmslab-root-ca.cer

CMS1b and CMS1c will have the exact same certificate files except that they will have cms1b.key/cms1b.cer or cms1c.key/cms1c.cer instead of cms1a.key/cms1a.cer.

This simply verifies that the files are actually there, it does not check the contents in any way. CMS does provide the pki inspect command to dump the contents of a file, which is useful for checking things like the expiration date or subjectAltName fields present, as well as the pki verify command to verify the certificate is signed by the CA and that the certificate bundle can be used to assert this.

Feel free to test this, for example, on cms1a, you could issue the command:

cms1a> pki verify cms1a.cer cms1abc.cer cmslab-root-ca.cer Success

If this succeeds, you know that the certificate you generated is signed by our CA certificate and is in fact correctly represented in the CA bundle.

It is also possible to verify that a certificate matches a given private key with the pki match command:

cms1a> pki match dbclusterclient.key dbclusterclient.cer Matching certificate and private key