Host key signing?

[hany]

I'm wondering if thought has been put into having ssh-cert-authority not only handle signing user keys, but also host keys? A similar challenge exists when one wants to sign host keys to allow for consistent known_hosts files. The CA's private key is needed to sign the public keys, and having the private key copied to the hosts is not desirable. The result is needing to copy public keys back and forth, which is part of what this project solves.

[bobveznat]

Thoughts, yes. In general I worry about authenticating systems on boot.

It's super simple to just auto-sign any host that boots. Unfortunately this is about as useful as simply typing "yes" when prompted by the ssh client.

In the old project, ssh-ca, there's a script I wrote to ease signing host certs: https://github.com/cloudtools/ssh-ca/blob/master/scripts/sign_host_key . On long-lived systems (there are very few of these) at my employer we've used this to sign the host certs.

To automate this we need authentication. We need something that can reliably know a host was supposed to boot, it actually booted, it booted the OS we wanted it to boot it has a key fingerprint of x and then when we generate and sign the host certificate against the host's actual private host key.

And that's a hard problem.

That said, that's getting a bit idealistic. Really what we should aim to do is improve, at least slightly, on current real world behavior of simply typing yes. If we could do simple things like parse EC2 console output to rip out the host key fingerprints and use that as our source of authentication that might be better than the current state of the world. The flow might be:

• Instance boots
• Instance asks ssh-cert-authority to sign its host cert
• ssh-cert-authority gets the system console log for the instance id referenced by the IP that connected to us
• If the fingerprints in the log match the fingerprints in the cert request then we sign it
• The host polls for its signed cert, downloads it and installs it.

Those are some of the ideas I have so far. Any different thoughts for how it might work?

[hany]

Hey @bobveznat, you raise some good points, and our intention was to perform some independent validation of the host and not blindly sign the certificates. In our case, we would have new cloud instances boot up, and in a cloud-init config perform the following:

1. Generate the host keys.
2. Call some HTTP service (such as ssh-cert-authority) and pass along the public key, along with the UUID of the instance, which can be fetched from cloud-init.
3. The HTTP service would independently verify the UUID of the instance with the cloud's API to ensure this is from a new server that is being built.
4. Sign the certificate and pass it back for the host to install.

Very similar to your thought process. Most clouds support the use of cloud-init, but not all allow direct programatic access to the console.

Thoughts?

[hany]

@bobveznat now that #6 is out the door, I wonder if we can revisit this particular issue. Being able ti sign host keys would be a huge boost in security for most ephemeral cloud environments, as we've become all to used to simply typing yes when SSH asks us to verify the host. Would it be possible to leverage the capabilities of #6 to sign host keys, leaving the task of verification of the requests to the implementors?

[bobveznat]

Yeah, I'm still thinking about this. Kind of waiting to hope I come up with a simple solution.

One avenue of thinking is using the instance identity document which work well for EC2 instances but no other CSP that I'm aware of (Though I've formally requested similar functionality from GCE). Are you an Amazon shop?

[hany]

@bobveznat we're actually a multi-cloud user, but primary use CloudStack. CloudStack has the same mechanism's as AWS' instance identity document. cloud-init is supported by CloudStack as well and AWS, and the request process can be kicked off from this.

However, I assume the line of thinking is the server process would accept these unique IDs, and perform some sort of independent lookup using the CSP's native APIs to validate the system?

Perhaps to keep the scope in check, the server can call out to an external script that is user-maintained to verify the information passed by the requester?