Boundary Client Agent
This feature requires HCP Boundary or Boundary Enterprise
Beta feature
Beta functionality is stable, but possibly incomplete and subject to change. We strongly discourage using beta features in production deployments of Boundary.
The Boundary Client Agent is Boundary's DNS daemon. When the Boundary Client Agent runs alongside an authenticated Boundary client, the agent establishes itself as the primary resolver on the system and intercepts DNS requests.
If you enter a hostname that matches a Boundary alias that the client is authorized to establish a session to, Boundary automatically generates the session and transparently proxies the connection on your behalf. If the Boundary Client Agent cannot find an alias, or if there are any issues with authentication, network connectivity, or latency, the Client Agent defers DNS resolution to the previously configured DNS resolvers.
Security
When you successfully authorize a session on a Boundary controller, the response includes a list of any brokered credentials, which include the decoded secrets. When the Boundary Client Agent receives a DNS request, Boundary creates a new session. An OS user can only connect to an authorized session managed by the Boundary Client Agent daemon if they are the same OS user that added the Boundary auth token used for authorizing the session.
Note
Currently, you cannot authenticate to multiple Boundary controllers at once. If you authenticate to a different Boundary controller, any existing sessions are terminated and any new transparent sessions would be established with the new controller.
The Boundary Client Agent stores the credentials and some other information related to the session in memory. The in-memory store removes the session information:
- when the session ends.
- if the auth token stored in the Boundary expires.
- if the current OS user authenticates with a different Boundary user.
- if the current OS user authenticates to a new Boundary controller.
- if the Boundary Client Agent is paused.
- if the Boundary Client Agent is terminated.
API requests are authenticated in the same way as session proxy access.
Credential brokering is supported for transparent sessions. A notification appears when you establish a session against a target that is configured with credential brokering. You can retrieve the credentials later using the following command:
Grants
The default grants that Boundary creates for anonymous and authenticated users are sufficient to get started with the Client Agent for the public beta. However, in a production scenario, you may want to provide the least amount of privileges necessary for users. For a Boundary user to be able to use the Client Agent to establish a transparent session, they must:
- be able to authenticate using an auth method.
- have read permissions for their auth token.
- have permission to establish a session to one or more targets.
You can use the following grant strings to grant those permissions:
HashiCorp highly recommends that you also grant users the permission to list resolvable aliases, as the Client Agent periodically fetches a list of aliases to match incoming DNS requests against. Without that permission, every DNS request on the system is sent to the Boundary controller, which can easily overwhelm it. You can use the following grant string to grant the permission to list resolvable aliases:
Configuration
The default configuration included with the Boundary Client Agent upon installation will be suitable for most users. If you want to make changes to the configuration, the configuration file is located in the following directory:
/Library/Application Support/HashiCorp/Boundary/boundary-client-agent.hcl
The configuration file contains the following fields:
alias_refresh_interval
- Specifies how often to refresh the alias cache. The default value is 1 minute.Example:
dns_request_timeout
- Specifies for how long the Client Agent DNS request handling, including any recursion, is allowed to run before it is canceled.Example:
interface_to_use
- Specifies the interface to use instead of the default.Example:
log_file
- Specifies where to write the Boundary Client Agent log file to.Example:
log_level
- Specifies the verbosity of the Client Agent logs.Example:
log_to_stdout
- Logs to STDOUT in addition to theboundary-client-agent.log
file.Example:
override_upstream_dns_servers
- Lists the DNS servers that should be used for recursing non-Boundary requests, overriding those configured on the system.Example:
state_file
- Specifies where to write the Boundary Client Agent state file to. This is an ephemeral file which is removed on successful shutdown.Example:
v4_prefix
- Specifies an alternate prefix to use for generating IPs. Currently must be between /8 and /16Example:
Change the configuration
Complete the following steps to change the configuration of the Client Agent:
As a privileged user, open the Boundary Client Agent configuration file in the editor of your choice. By default, it is located in the following directory:
/Library/Application Support/HashiCorp/Boundary/boundary-client-agent.hcl
Change the configuration settings, and save the file.
Note
You must restart the Client Agent to update some configuration settings. However, when you restart the Client Agent, it closes any existing sessions. Other configuration settings can be updated by only reloading the configuration file, which does not affect any existing sessions.
Either reload the configuration file or restart the Client Agent.
You can change the following configuration values by reloading the configuration file, which will not disrupt any existing sessions:
Run the following command to reload the configuration file:
If you want to update another configuration value, you can restart the Client Agent using the following commands, however it will close any existing sessions:
Manage the Client Agent
Refer to the following sections for more information about managing the Client Agent. You can monitor the Client Agent's status and retrieve information about any transparent sessions. If you want to temporarily defer DNS resolution to any previously configured DNS resolvers, you can pause the Client Agent. You can also disable the Client Agent, if you no longer want to use it for DNS resolution.
Monitor status and sessions
You can check the status of the Client Agent to ensure it is running. Use the following command to check the Client Agent's status:
You can retrieve information about the sessions that the Client Agent is managing. Use the following command to list any sessions currently being managed by the Client Agent, as well as any brokered credentials for those sessions:
Note that this command does not list sessions that are not managed by the Client Agent. Use boundary sessions list -recursive
to see all sessions.
Pause the Client Agent
You can temporarily disable the Boundary Client Agent by pausing it with the following command:
When the Client Agent is paused, it does not intercept any DNS requests, and you are unable to use transparent sessions.
To resume the Client Agent, use the following command:
Disable the Client Agent
If you want to disable the Boundary Client Agent, you can stop it with the following commands:
Troubleshooting
The following sections can help you to troubleshoot the Client Agent's behavior. You should proceed through these steps from top to bottom.
Check the status of the Client Agent
If you experience unexpected behavior, you should first check on the status of the Client Agent. You can check the status using the Boundary CLI or the Desktop Client. To check the Client Agent status through the Boundary CLI, use the following command:
In the Desktop Client, you can find the status of the Client Agent by navigating to Settings, and then scrolling to the Boundary Client Agent section.
The status command includes various information about the Client Agent, including the runtime status. In this example, the runtime status is "running". If the status is "paused", the Client Agent is not currently intercepting DNS requests and must be resumed. Users can pause the Client Agent, and it will also pause itself if it detects a large number of network failures in a short period of time.
The status also allows you to see whether the current user is authenticated. If the response looks like the example above, including showing an auth token ID and expiration, your current user is authenticated. If not, you may need to first authenticate to the Client Agent using the CLI or Desktop Client.
The status also sometimes contains a list of errors that have been encountered by the Client Agent. The list is ordered by most recent first. These errors can help you understand why the Client Agent may not be behaving as expected. Please see the section below on commonly seen errors to help diagnose specific errors. Note that this list of errors will not be cleared until the next reboot, so it may not necessarily be a sign of something being wrong.
If the status command returns an error, the Client Agent may not be running. You can attempt to start the Client Agent using the following commands:
Resume the Client Agent
You can resume the Client Agent using either the Boundary CLI or the Desktop Client. In the CLI, run the following command to resume the Client Agent:
In the Desktop Client, you can resume the Client Agent by selecting the Resume button in the Boundary Client Agent section of the settings. Once the Client Agent has resumed, test if it has started working as expected again.
Inspect the log file
If you are not able to diagnose the problem by looking at the status or resuming the Client Agent, another step can be to inspect the log file produced by the Client Agent.
The log file should be located in /Library/Application Support/HashiCorp/Boundary/boundary-client-agent.log
.
Once you have found the log file, you can look through it to see if you can understand why the Client Agent is not working as expected. The list below provides some common errors and explanations.
It may be necessary to increase the logging verbosity of the Client Agent.
You can increase the verbosity by setting the log_level
option in the configuration file to "DEBUG"
.
See the section on changing the configuration for more information.
Establish the behavior of the local DNS configuration
The Client Agent works by intercepting DNS requests before they are sent to your regular DNS server. If the DNS requests on your system are not sent to the right place, or they are not being answered appropriately, transparent sessions will not work.
You can use the nslookup
command to understand where the DNS requests are being sent.
Start by sending a DNS request for hashicorp.com
:
The important part here is the Server
field, which contains an IP in the CGNAT range (from 100.64.0.0
to 100.127.255.255
).
This is a good indication that the Client Agent DNS server is being used as expected.
Next, you can try to make a DNS request to an alias that you expect to work. The following example makes a DNS request to an alias with a value of mytarget.boundary.dev
:
You can tell two things from this:
- The Client Agent is likely able to intercept the DNS request, because the server is a local IPv6 address in the ULA range. Both an IPv4 CGNAT range or IPv6 ULA range IP address are indications of this.
- The Client Agent is able to identify
mytarget.boundary.dev
as an alias with a target that the requesting user is authorized to connect to, because it responded with a valid DNS response pointing to a local IPv4 address in the CGNAT range. Similarly to above, the IP address in the response may also be an IPv6 address in the ULA range.
If you do not see this kind of response, it may be that the alias you are trying to connect to doesn't exist, or your user is not authorized to connect to it. Double check that you are using the correct alias and that your user is authorized to connect to it.
Flush OS DNS cache
If you still do not see the expected behavior, it can be useful to flush the operating system's DNS cache. The exact steps depend on the operating system you use:
After you flush the DNS cache, try connecting to the alias again, or repeat the steps above.
If you are still not able to understand what is wrong, submit a support ticket. Include the boundary client-agent status
output and the log file in the ticket.
Commonly seen errors
Refer to the following commonly seen errors for more information about their possible causes and resolutions.
nodename nor servname provided, or not known / No such host is known
This is a generic error for a failed DNS resolution. It can mean a number of different things:
- The alias doesn't exist or is misspelled.
- Your user isn't authenticated or doesn't have permission to connect to the target.
- The Client Agent is not able to intercept DNS requests, it could shut down or paused.
- The OS DNS cache is interfering with the operation of the Client Agent.
- The Client Agent may not yet know about the alias. It takes around 2 minutes for the Client Agent to learn about new aliases.
Follow the troubleshooting steps above to resolve the issue.
failed to listen for DNS on either IPv4 or IPv6
This error happens when some other application on the local machine occupies the ports used by the Boundary Client Agent. The Client Agent requires access to port 53 for IPv4 and IPv6, both UDP and TCP. Diagnosing what causes the error differs per operating system:
As a privileged user, you can use the lsof
program to find what applications are occupying a port, for example:
If anything is occupying port 53, you may need to terminate the application before the Client Agent is able to start.
Applications that make use of the Apple Virtualization Framework are known to sometimes occupy this port under
the name _mdnsresponder
. If you have any virtualization software, you may need to turn it off before using
the Client Agent.
Once you have identified which other software is using the port, you can stop it and try to start the Client Agent again.
failed to refresh alias cache: error="fetching resolvable aliases: error performing client request during List call"
This usually implies that there is a problem reaching the internet or the Boundary controller. The error is related to the periodic updating of aliases used by the Client Agent to know whether a DNS request matches an alias or not.
HashiCorp recommends that you pause the Client Agent and examine the status and logs for further errors:
Follow the troubleshooting steps to understand why the Client Agent is not able to reach the controller.
Conflicting software
Some software is known to cause conflicts with the Boundary Client Agent. The following sections are an incomplete list of potential conflicts and any available workarounds for issues.
Docker Desktop (MacOS)
Docker Desktop sometimes creates a local DNS listener that prevents the Client Agent from running.
If you run Docker Desktop 4.26 or later, you must clear the Use kernel networking for UDP
option.
Otherwise, the Client Agent refuses to start.
Palo Alto Networking Global Protect VPN
If you are unable to establish a transparent session while using the Palo Alto Networking Global Protect VPN, you may need to explicitly specify a network interface and the upstream DNS server(s) to use.
By default, the Client Agent reads the primary network interface's DNS server configuration and uses that information to resolve domains that are not configured as aliases in Boundary. If the VPN configuration includes custom DNS servers, this information may not be available to the Client Agent, so you must explicitly specify the DNS server(s) to use.
To configure the DNS server(s) to use, use the override_upstream_dns_servers
configuration option:
Note
The override_upstream_dns_servers
is used for all non-Boundary DNS requests.
If you only provide the VPN DNS servers, the Client Agent will not be able to resolve any DNS requests when the VPN is not active.
Primary network interfaces
By default, the Client Agent creates IPs on the primary network interface to serve its DNS server. Refer to the tabs below for possible conflicts for each supported operating system.
When you run the Client Agent alongside the PAN-GP VPN, the primary network interface will likely be set to a tun
type interface, which the Client Agent cannot use for its IP addresses.
You may see errors such as the following in the boundary-client-agent.log
file or the boundary client-agent status
command response:
To work around the default tun
interface, you must provide an explicit network interface using the interface_to_use
configuration option. For example:
The interface_to_use
option allows the Client Agent to create the IPs it needs to serve the DNS server and proxy traffic.
You must restart the Client Agent for it to update its configuration with the new setting.
Cloudflare WARP client
The Cloudflare WARP client uses a local DNS server to direct traffic. It has built-in checks to prevent it from being run alongside other software that uses the same mechanism. This includes the Boundary Client Agent. If you try to use the Client Agent with the Cloudflare WARP client, it may work, or you may see an error like this one:
You can still install both the Cloudflare WARP client and the Boundary Client Agent on the same machine. As long as you don't run both at the same time, they should work as expected.
Uninstall the Client Agent on Mac
If you used the Mac installer, you can run /Library/Application Support/HashiCorp/Boundary Uninstaller.app
to uninstall Boundary.
The uninstaller removes any installed components, including the Desktop client, CLI, and the Boundary Client Agent.
More information
Refer to the following topics for more information: