SSH Key Management for Cloud Teams: When to Go Keyless

Count the public keys in the authorized_keys files across your servers, then compare that number with the people who should have access. For most teams the first number is bigger. Some keys belong to people who have left, some were copied to a second laptop, and one was generated for a deployment pipeline years ago and never touched again.

SSH key management is how you get those numbers to match and keep them matched. For a small team, disciplined key hygiene is enough. Past a certain size, the better answer is keyless SSH: short-lived certificates that expire on their own, so there are no long-lived keys left to track. If you have not checked recently, our 30-minute server access audit is a good first pass, and the glossary explains how SSH key pairs work.

Why SSH keys become a problem as teams grow

SSH (Secure Shell) keys are strong credentials. The trouble is everything around them.

They never expire. A public key in authorized_keys works until someone removes it. Standard SSH key authentication has no built-in lifetime.

They are not tied to an identity. The server knows a key, not a person. If a key is copied, the server cannot tell the difference.

They spread. Every server keeps its own list. Ten engineers and forty servers can mean hundreds of entries, maintained by hand or by a script someone wrote once.

Offboarding misses them. Disabling someone's single sign-on (SSO) account does nothing to the key sitting in authorized_keys on a database server.

SSH key management done properly

If you are staying with keys, these practices close most of the gaps.

Keep an inventory. Know which keys are trusted on which servers, and who owns each one. Running ssh-keygen -lf ~/.ssh/authorized_keys prints fingerprints you can match against a register.

One key per person per device. Shared team keys make revocation impossible without locking everyone out.

Use modern key types and passphrases. Ed25519 keys with a passphrase, held in an agent, or hardware-backed keys where your team can support them.

Turn off what you do not need. Disable password authentication and direct root login in sshd_config.

Give machines their own identities. A deployment pipeline should never use an engineer's key, and its own key should be limited to the commands it needs.

Rotate and remove on a schedule. Set a rotation interval, and make key removal a named step in offboarding, with one person accountable for it.

If you would rather hand the operating system side of this to engineers, OS patching and hardening is part of the managed services included on every InMotion Cloud plan.

This approach works, and plenty of teams run it well. It also depends on a list that people keep current by hand, which is the part that tends to slip.

What keyless SSH actually means

Keyless SSH does not remove cryptography. It removes long-lived keys as the thing you manage. The usual mechanism is the SSH certificate, which OpenSSH has supported since version 5.4, released in 2010.

It works like this:

1. You create a certificate authority (CA): a signing key kept somewhere well protected.
2. Each server trusts that CA through one line in sshd_config: TrustedUserCAKeys /etc/ssh/user_ca.pub.
3. When an engineer needs access, they sign in, usually through your identity provider, and the CA signs their public key into a certificate valid for a set period, such as eight hours.
4. The server accepts the certificate because it trusts the CA. There is no per-server authorized_keys entry for that person.
5. When the certificate expires, access ends. Nobody has to remove anything.

Certificates also carry principals, which control which accounts the holder can log in as, and they can carry restrictions such as a forced command. You can revoke one before it expires with a key revocation list, though short lifetimes mean you rarely need to.

The same idea works in the other direction with host certificates. Servers present a certificate signed by your CA, so engineers stop accepting "are you sure you want to continue connecting" prompts they have no way to verify.

Keys or certificates: which fits your team

No fixed headcount makes certificates mandatory, but the trade-off is fairly predictable.

Stay with keys if you have a handful of engineers, a small number of servers, and someone who reliably owns the inventory.

Move to certificates when people join and leave often, when automation creates and destroys servers, or when an auditor asks you to show who had access to a system on a given date.

Use short-lived certificates for pipelines if your continuous integration (CI) provider can issue identity tokens. The pipeline proves who it is, receives a certificate valid for minutes, and no long-lived secret sits in your CI settings.

The cost of certificates is the CA itself. It becomes one of the most sensitive systems you run, and someone has to operate it, back it up and plan for replacing it.

What changes on a private cloud

On OpenStack, including InMotion Cloud, you choose a key pair when you launch an instance, and it is placed on the server at first boot. Our guides cover creating and managing SSH key pairs in the dashboard and importing an existing key.

That injection happens once. Adding, rotating or removing keys afterwards happens inside the server, by you or your configuration management. If you move to certificates, the cleanest route is to bake the TrustedUserCAKeys line and the CA public key into your base image, so every new instance trusts the CA from first boot.

SSH is also only one part of access on a private cloud. Our article on identity and access management for private cloud covers the dashboard and API side.

What InMotion Cloud does, and doesn't

InMotion Cloud gives you key pair management at launch through the OpenStack dashboard, and our engineers can help harden SSH configuration as part of managed operating system hardening.

We do not run an SSH certificate authority for you or issue certificates on your behalf. If you go keyless, you choose and operate the CA, or the access tool that provides one, and we can help configure your servers to trust it.

Summary

SSH keys cause trouble as a team grows because they never expire and nothing ties them to a person. An inventory, one key per person, separate machine identities and disciplined offboarding fix most of that. When the list grows too large to keep accurate by hand, short-lived SSH certificates remove the list altogether.

Frequently asked questions

What is keyless SSH? Logging in with a short-lived certificate issued after you authenticate, instead of a long-lived key listed on each server. The certificate still involves a key pair, but there is nothing long-lived to manage.

Are SSH certificates the same as TLS certificates? No. OpenSSH certificates use their own format and are issued by an SSH certificate authority you control.

How often should SSH keys be rotated? Set an interval that fits your risk and audit requirements, and rotate at once when a device is lost or someone leaves.

Can I use SSH certificates on OpenStack? Yes. Configure your servers to trust your CA, ideally in the base image, and they will accept the certificates it issues.

What happens if the SSH certificate authority is compromised? An attacker could issue valid certificates, so protect the CA key closely and plan in advance how you would replace it and reconfigure servers.