SSH Credential Injection with HCP Boundary
Boundary 0.10 adds support for credential management of SSH keys, starting with credential brokering. HCP Boundary introduces credential injection: the ability to inject SSH credentials into sessions, particularly via Vault. Currently users must bring their own Vault deployment to HCP Boundary using a public endpoint.
This tutorial will demonstrate SSH credential injection into an SSH server configured using Docker. Learners may also bring their own target to use for this tutorial.
Tutorial overview
- Prerequisites
- Background
- Set up an SSH target
- Set up Vault
- Set up Boundary
- Inject credentials into sessions
Prerequisites
This tutorial assumes the learner has completed the HCP Boundary Getting Started tutorials. The learner should have a working Boundary cluster and org running on HCP.
Docker is installed (Note: Learners may also bring their own target, instead of using Docker)
A Boundary binary greater than 0.10.0 in your
PATHA Vault binary greater than 1.7.0 in your
PATHInstalling the Boundary Desktop App provides an optional workflow at the end of this tutorial. The 1.2.0 version or above is required for Vault support.
This tutorial assumes basic knowledge of using Vault, including running a development server and managing policies, roles, and tokens. If you are new to using Vault, complete the Getting Started with Vault quick start tutorials before integrating Vault with Boundary.
This tutorial also extends the workflow for credential brokering via Vault. If new to this process, we recommend completing the Vault Credential Brokering Quickstart tutorial before moving forward.
Credential injection background
Boundary is a distributed system for managing identity-based access to computing resources. Core to Boundary are controllers and workers which, from a resource point of view, are referred to as servers. Controllers are responsible for authenticating and authorizing users, as well as serving user API requests for processes like session initiation. They also assign jobs to workers, such as session handling, session recording, and parsing. Workers are responsible for session proxying, as well as other jobs that involve storage.
SSH, also known as Secure Shell, is arguably the de facto tool for remote administration, enabling end users and administrators with a simple, powerful and secure access method. SSH offers a variety of benefits and configurations, such as:
- Encryption
- Support for multiple keys types and sizes
- Built in key exchange
- Ability to create a tunnel and spawn multiple channels
Boundary seamlessly provides authenticated users access to remote machines via their SSH client, regardless of the authentication mechanisms implemented by their administrators.
Previously Boundary supported credential brokering, where Boundary controllers check out credentials from Vault and return them back to users and clients. In this workflow, Boundary acts as a broker of the credential.
Brokering lacks the ability to hide credentials from clients. For credential injection, controllers instead return credentials to Boundary workers and create a session to a target where the user/client never has access to the credential. This workflow is called credential injection because the credential is injected into a worker’s session, instead of being passed back to the client.
Set up an SSH target
This tutorial assumes the reader has installed Boundary 0.10.0 or above, and a Vault binary greater than 1.7.0. Vault can be run in development mode locally, as demonstrated in this tutorial, or by supplying a Vault public address to HCP Boundary.
First, you are going to set up a target which will be used to test the credential injection using Vault.
Note: Alternatively, you can bring your own target to test the credential injection. Credential injection is currently supported for user/pass and sshpass credentials. If bringing your own target, skip to the Set up Vault section.
Docker is used to deploy an openssh-server target for testing credential injection.
For the purposes of this tutorial, create an ssh keypair that will be used to log in to the openssh container.
First, open a terminal session and create a new directory to store the keypair
in. This tutorial creates the openssh directory at /, but you can place this
directory elsewhere if desired.
Next, create a new ssh keypair.
Press enter twice when prompted for a passphrase, or supply one if desired.
The above command will create the keypair in the /openssh/ directory at
/openssh/openssh-key and /openssh/openssh-key.pub. Update this path if you
wish to store the key elsewhere. Note the path to the keypair for later.
Ensure Docker is running, and then deploy the openssh container. Supply the
path to the /openssh/ directory created earlier after the --volume=
option. This will mount the folder within the docker container, making the
public key file available at /keys/.
Note that the username for the openssh server is admin, and the target is
available on your localhost at port 2222 (127.0.0.1:2222).
Leave this container running for the duration of the tutorial.
Set up Vault
Note: This tutorial recommends running a Vault dev server to test credential injection. Vault environments on a public subnet can also be used with HCP Boundary. If bringing your own Vault instance, ensure that public connections are configured (refer to the Create a Vault Cluster on HCP tutorial), and then skip the running Vault in dev mode section of this tutorial.
Credential injection with Vault can be set up with a Vault dev server or using HCP Vault.
Open a new terminal session and run Vault in development mode.
Leave Vault running in dev mode and open a new terminal window.
Lastly, export the required Vault environment variables for the address and token.
Define the controller policy
As described in the OSS Vault Credential Brokering Quickstart tutorial, the following Vault policy must be defined for the Boundary controller to support credential brokering.
Create a new file named boundary-controller-policy.hcl and copy the following
policy into it.
Save this file, and write the controller policy to Vault.
Define the KV policy
A policy must also be defined for the key-value secrets that will be brokered via Vault.
Create a new file named kv-policy.hcl and copy the following policy into it.
Save this file, and write the policy to Vault.
Lastly, create a new Vault KV credential at the secret/my-secret path and
provide the username and private key for the openssh-server target machine you
are connecting to. Ensure the path to the private key is entered after the @
symbol.
Note: If you brought your own target to test credential brokering
against, enter the target’s username after username and path to the private
key after the @.
Additionally, brokered credentials can also be attached to SSH targets. These credentials will be brokered to the end user after injection. This is useful in the case that a user needs access to application credentials on the target after the SSH credentials are successfully injected.
Create a second KV credential at the secret/my-app-secret path for
brokering. This example will user a simple username and password that will be
passed back to the authenticated user.
Create a Vault token
Now create a Vault token.
Example:
Copy the .auth.client_token value and export it as the
$CRED_STORE_TOKEN variable.
Note: If you plan on using the Admin Console UI to connect, save this value for creating a credential store later on.
Tip: If you have the jq utility installed, this can be performed in a single step with this command:
Set up Boundary
Start by logging in to HCP Boundary within the terminal.
Export the BOUNDARY_ADDR and BOUNDARY_AUTH_METHOD_ID environment variables. These
values are gathered from the HCP Boundary Admin Console, as demonstrated in the
Get Started with HCP
Boundary
tutorials.
Log in to the CLI as an admin user, or a user with privileges to manage
resources in the global scope. Enter your admin username for login-name.
Enter password at the Please enter the password (it will be hidden): prompt.
You will also need an ORG_ID to create the SSH test project within. You may
create a new global scope for this, or utilize an existing scope used for
testing.
If creating a new org for testing, execute the following:
Whether you create a new org or not, be sure to export the org ID.
Set up a new project
Create a new project scope within Boundary. You will need an org ID to create
the new project within. To view your existing orgs, execute boundary scopes
list -recursive.
Note: You may use an existing org scope, or create a new org scope for the ssh-project. Refer to the Manage Scopes tutorial to learn more about creating and managing scopes.
Copy the ID from the output and export it as the $PROJECT_ID variable.
Create a credential store
Next, create a new credential store within Boundary using the new token. The
vault credential store type is used to integrate with Vault, but static
credential stores can also be used with credential injection. Note: If using HCP Vault,
provide the vault namespace while creating the credential store.
If using HCP Vault,
Example output:
Copy the ID from the output and export it as the $CRED_STORE_ID variable.
Tip: If you have the jq utility installed, this can be performed in a single step with this command:
Create credential library
Create a new credential library of type ssh_private_key within Boundary
using the credential store ID and passing the vault-path of my-secret.
Example output:
Copy the ID from the output and export it as the $CRED_LIB_SSH_ID
variable.
Tip: If you have the jq utility installed, this can be performed in a single step with this command:
Additionally, create another credential library of type username_password for
the my-app-secret credential.
Example output:
Copy the ID from the output and export it as the $CRED_LIB_APP_ID
variable.
Tip: If you have the jq utility installed, this can be performed in a single step with this command:
Create an SSH target
First, create a new static host catalog.
Example output:
Export the new host catalog ID as the HOST_CATALOG_ID environment variable
Now create a new static host.
Note: Learners may also bring their own target to test credential
injection. If bringing your own target, enter the connection address for the
-address option. The connection port is supplied when creating a target later
on.
Example output:
Export the openssh host ID as the HOST_ID environment variable.
Then create a static host set.
Example output:
Export the host set ID as the HOST_SET_ID environment variable.
Add the new host to the host set.
With the host set up, create a new target of type ssh and set its default
port to 2222.
Note: Learners may also bring their own target to test credential
injection. If bringing your own target, enter the SSH connection port (likely
22 for SSH) for the -default-port option.
Example output:
Copy the ID from the output and export it as the $TARGET_ID variable.
Tip: If you have the jq utility installed, this can be performed in a single step with this command:
Add the target to the host set.
Associate the SSH target with the SSH credential library for credential injection.
Inject credentials into sessions
Now you are ready to inject credentials directly into the shell session. You can
accomplish this using the boundary connect command, which has a helper called
ssh.
Enter yes when prompted to establish the connection.
When finished, the user can close the connection to the server using exit, or
the session can be canceled directly from Boundary in a new terminal window:
Copy the session ID and cancel the session:
You can verify the session was canceled by listing the sessions again and
noticing the terminated Status:
In addition to injecting SSH credentials, brokered credentials can also be passed to the user if they are attached to the target. This can be useful when the user needs access to application credentials once they are connected to the target.
Attach the username_password application credentials to the target by
supplying the $CRED_LIB_APP_ID to the -brokered-credential-source option.
Notice that the target now has both injected and brokered credential sources.
Connect to the open-ssh target again.
After credential injection is complete, the brokered credentials are also passed on the CLI, enabling the user to utilize the credentials within their session.
Note: There are cases in which targets might have multiple SSH keys or username_password credentials attached as injected application credential sources. SSH supports multiple credential sources, and Boundary will try them until it finds a usable credential when attempting a connection. This scenario is useful when rotating private keys. Both the old key and the new key can be attached to a target simultaneously, and Boundary will try both of them, stopping if one is successful. This facilitates key rotation without downtime.
Inject credentials with the Desktop App
To log into Boundary using the Desktop App, the BOUNDARY_ADDR (Boundary
cluster address) and BOUNDARY_AUTH_METHOD_ID (user Auth Method ID) values must be
gathered from the HCP Boundary Admin Console, as demonstrated in the HCP
Boundary Getting Started
tutorial.
Open the Boundary desktop app.
Enter the Boundary cluster URL (for example,
https://ffee961b-5fd8-4e68-ba1d-2bbb487b576e.boundary.hashicorp.cloud) and
click Submit.
Authenticate using your HCP Boundary user credentials.
Under the Targets page, notice the target details for ssh-target.
Click Connect to initiate a session.
The Successfully Connected page will display the target ID (Target Connection details) and Proxy URL.
Open the raw API output.
Scrolling through the raw API output, you will find the username_password
brokered application credentials. The user should note these credentials when
using the Desktop app to establish connections.
To start a session, open your terminal or SSH client. A session can be started using SSH and the Proxy URL from the Boundary desktop app.
For the openssh docker target, connect on 127.0.0.1 and provide the proxy
port using the -p option. Enter yes when prompted to establish a connection.
When using the Desktop app to connect, the brokered credential are displayed in the raw API output, not upon connection.
When finished, the user can close the connection to the server by entering
exit, or the session can be canceled directly from the Boundary desktop app
under the Sessions view.
Cleanup and teardown
Locate the terminal session used to execute the vault dev server command, and
execute ctrl+c to stop Vault.
Unset the environment variables used in any active terminal windows for this tutorial.
Destroy the openssh-server container created for the tutorial.
Check your work by executing docker ps and ensure there are no more containers
from the tutorial leftover. If unexpected containers still exist, execute
docker rm -f <CONTAINER_ID> against each to remove them.