HashiConf Last chance to register for our cloud conference October 10-12. Register today
Vault
Vault Home

You are viewing documentation for version v1.9.x. View latest version.

Frequently Asked Questions (FAQ)

This FAQ section contains frequently asked questions about the client count feature.

Q: What is a client?

Clients are unique applications, services, or users that authenticate to a HashiCorp Vault cluster. For billing and consumption, only unique and active clients during the billing period (monthly in the case of HCP and annual in the case of self-managed Vault) count towards totals. Each client is counted just once within a billing period, regardless of how many times it's been active. When a client authenticates to a cluster, those clients have unlimited access to that cluster for the remainder of the billing period. The client metric is a combination of active identity entities and active non-entity tokens. To learn more, refer to the documentation on What is a Client?.

Note: For KMIP, a different metric is used to compute clients, as described in the Vault Usage Metrics documentation under the KMIP Client metrics section.

Q: Where can I learn more about Vault clients?

Refer to the table below for documentation resources.

ResourceDescription
What is a Client?Provides a conceptual overview of Vault client
Usage Metrics UIProvides an overview of the client count dashboard and how to use it
Client Count APIProvides information about the client count API endpoints
Vault Auditor ToolProvides a walkthrough on how to use the vault-auditor tool to extract metrics from the server audit device logs

Q: What is the difference between a direct entity and a non-entity token?

While the definition of clients appears to be simple on the surface, there are many nuances involved in the computation of clients. As mentioned, clients are unique applications, services, and/or users that authenticate to a Vault cluster. When anything authenticates to Vault, it is associated with a unique identity entity within the Vault Identity system. The name reported to the identity systems by the different types of authentication methods varies, and each entity is created or verified during authorization.

One thing to note is that Vault clients are a combination of active identities as well as non-entity tokens. Identity entities are unique users, and when identities authenticate to Vault, corresponding tokens are generated. However, there are some situations in which tokens are generated without corresponding identities (e.g., when using the token auth method to create a token for someone else whose identity is unknown). As such, these non-entity tokens also represent users, and are counted towards the overall client aggregates. Here are some situations in which non-entity tokens get created within Vault.

  • Tokens within Vault are the core method for authentication. You can use Tokens to authenticate directly, or use the auth methods to dynamically generate tokens based on external identities.
  • There are scenarios where tokens are created outside of the identity system without an associated entity. For this reason, unique identity entities alone cannot always add up to the total unique authentications made to Vault over a stipulated time period.
  • In a scenario where tokens are created outside of the identity system, these tokens are considered clients. Note that it should be rare for production usage to have any tokens created outside any identity systems.
  • There are a few ways of creating tokens without entities: Token Roles, Token Create APIs, Wrapping Tokens, and Control Groups. For more information, refer to the What is a Client? documentation.

Client counts are not computed solely using a combination of unique identity entities within Vault but also computed using a combination of unique identity entities and non-entity tokens.

Q: Which Vault version reflects the most accurate client counts?

Although client counts have been available via the usage metrics UI since Vault 1.6 and the Vault auditor tool was available around the same time, we have since made improvements to the Vault client count computation logic in newer versions of Vault. These changes are reflected only in the usage metrics UI (and the corresponding internal API leveraged by the UI) and not in the auditor tool. The initial version and improvements made after that with their corresponding versions are:

API/UI:

  • Vault 1.6: Introduction of client counts in the usage metrics UI
  • Vault 1.8:
    • Eliminated wrapped tokens and control groups from client count, thereby reducing the non-entity token count. Previously, the creation and usage of control groups and wrapping tokens affected the client count each time the response is read (in the case of a wrapping token) and each time a control group was created (a non-entity token was created)
    • Changed the logic of counting of active identity entities on usage instead of at create time, resulting in more accurate client counts
  • Vault 1.9:
    • Changed the non-entity token computation logic to deduplicate non-entity tokens, reducing the overall client count. Moving forward, non-entity tokens, where there is no entity to map tokens, Vault will use the contents of the token to generate a unique client identifier based on the namespace ID and associated policies. The clientID will prevent duplicating the same token in the overall client count when the token is used again during the billing period.
    • Changed the tracking of non-entity tokens to complete on access instead of creation.
    • Changed the computation logic to not include root tokens in the client count aggregate.
    • Changed the local auth mount computation logic such that local auth mounts count towards clients but not as non-entity tokens. Prior to Vault 1.9, local auth mounts counted towards non-entity tokens. Refer to the What is a Client? documentation to learn more.
    • Added ability to display clients per namespace (top 10, descending order) in the UI and export data for all namespaces. Prior to Vault 1.9, you could not view view the split of clients per namespace on the UI, nor could you export this data via the UI.
    • Added ability to display clients earlier than a month (within ten minutes of enabling the feature) in the UI. Prior to Vault 1.9, after enabling the counting of clients, you had to wait for a month to view the client aggregates in the UI.

Auditor tool:

  • This tool is independent of the Vault binary and the version run by the customer. The tool can be used for versions 1.3-1.5 (tested) and prior versions as early as Vault 1.0. The auditor tool was introduced in Vault 1.6.
  • In Vault 1.7, we introduced KMIP clients to this auditor tool
  • This tool does not contain any of the updates made to the client count computation logic that the API/UI have had since Vault 1.8
  • In the future, we plan to deprecate this tool

The latest GA version of the Vault binary contains the most updated version of the client count computation logic. However, it’s important to note that even if one upgrades to the latest version and the billing period falls on either side of the upgrade time, the compute logic may be different across the billing period. For more details, refer to the question If I migrate from Vault 1.8 to 1.9, how will the changes to non-entity token logic and local auth mount made in Vault 1.9 affect the clients created prior to the migration?.

Q: For customers using older versions of Vault 1.6, what’s the best way to compute clients?

The Vault auditor tool was built to compute clients for Vault versions older than Vault 1.6. It has been tested for versions 1.3 to 1.5 but should work for earlier versions, such as Vault 1.0, although not officially tested. To use the Vault auditor tool, customers should have audit logs for the billing period for computed client counts. You can also set a specific date range in the auditor tool.

The auditor tool is separate from the Vault binary. It does not include the latest updates for the usage metrics UI/API (e.g., it does not contain changes made to the deduplication of non-entity tokens logic). Currently, there are no plans to augment the capabilities of the auditor tool.

Q: For customers using newer versions than Vault 1.6, what’s the best way to compute clients?

The usage metrics UI that leverages the internal client count API is the best way to compute clients for versions greater than or equal to Vault 1.6. It’s important to note that the upgrade should have occurred before the ‘billing period’ started. Otherwise, it’s best to use the auditor tool to compute clients because the usage metrics UI will not reflect all clients for the billing period; it will only reflect active unique clients since the upgrade.

Q: Why do we have two different tools (auditor tool and UI/API) to compute clients? Do we plan to deprecate one in the future?

Not all customers may be on a version greater than Vault version 1.6 that leverages the client count API to display clients via the UI. The auditor tool is available for customers running older Vault versions to compute client counts. The auditor tool does not contain client count computation logic updates (e.g., non-entity token computation logic made in Vault 1.9). In the future, we will deprecate the auditor tool.

Q: How can I compute KMIP clients for Vault?

As of Vault 1.9, KMIP clients are not available via the usage metrics UI or the client count API; they are provided via the auditor tool. To learn more, refer to the Vault Usage Metrics documentation.

Q: Why do the Vault auditor tool and the usage metrics UI show me different results for the total number of clients?

For versions prior to Vault 1.7, the auditor tool and usage metrics UI should typically result in the same number of total unique clients for the billing period, unless there is some discrepancy in the underlying data represented by both or due to some other error. For example:

  • If there is a discrepancy between the dates selected in the two tools, then the clients reflected in the results may be different
  • If the auditor tool does not leverage audit data for the billing period, the auditor results may be incorrect
  • If the Vault upgrade made to a version that computes clients in the usage metrics UI (any version greater than Vault 1.6) happened during the billing period, then the results shown in the usage metrics UI will only reflect the period since the upgrade occurred

However, there is one fine detail to keep in mind: the usage metrics UI only allows selection of billing periods from the start of a month to the end of a month (not mid-month). If the dates represented by the audit logs fed into the auditor tool begin and end mid-month, this may result in a subtle change of client counts between the two tools.

For newer versions of Vault 1.8, the API/UI for client counts was updated to reflect count clients more accurately. These changes were not made in the auditor tool and may cause a difference in the results generated by the two tools. For more information on the specific differences, refer to the question Which Vault version reflects the most accurate count of clients within Vault?.

Q: When I upgrade to a version of Vault that's greater Vault 1.6, will the clients be available for the entire history of the billing period, or only available after the upgrade occurred during the billing period?

The client counts will only be available after the upgrade occurs. For the complete billing period data, it’s preferable to refer to the auditor tool. However, keep in mind that since Vault 1.8, we made improvements to the client count API/UI that may cause mismatched results from the auditor tool. For more details, refer to the question If I migrate from Vault 1.8 to 1.9, how will the changes to non-entity token logic and local auth mount made in Vault 1.9 affect the clients created prior to the migration?. A workaround is to leverage the results from the UI/API (if on a newer version greater than Vault 1.8) instead of the auditor tool, and extrapolate the clients for the available period to the billing period.

Q: If I upgrade from Vault 1.8 to 1.9, how will the changes to non-entity token logic and local auth mount made in Vault 1.9 affect the clients created prior to the upgrade?

If you have a non-entity token for a fragment pre-Vault 1.9 version and then use the same token post-Vault 1.9 version, it will be counted again. However, for post-upgrade, the token will have an ID associated with it. From there, the subsequent uses of the token will not be counted again, as the token is tracked with the unique clientID. Hence, only for the period post the upgrade, the new deduplication logic for non-entity tokens are accounted for.

As for the local auth mounts, the tokens issued by the local auth mounts pre-Vault 1.9 version will be non-entity-tokens and counted as clients - 1 for each token. After the upgrade to Vault 1.9, the older tokens will continue to be counted as they were counted before, but the newer tokens issued will be part of entities and counted as far lesser clients (equal to the number of entities). To learn more, refer to the documentation on What is a Client?.

Q: Post Vault 1.9, will the clientID be viewable via the audit logs when non-entity tokens are used?

Yes, beginning with Vault 1.9, audit logs have a new field called clientID, which is the entity ID or the computed client ID of the corresponding non-entity token.

Q: What happens if audit logs are unreadable for use by the Vault auditor tool?

This issue may arise if the logs are too large and cannot be read, or you are running an older version than Vault 1.6. We encourage you to upgrade to a newer version of Vault so you can use the API/UI to compute clients. After the upgrade, even if you have only limited historical data on clients (since the upgrade), that data could be extrapolated for future client prediction. Account teams need to look into this on a case-by-case basis.

Q: What does the usage metrics UI look like for Vault 1.9?

In Vault 1.9, the client count dashboard provides two separate tabs: the Current month and the Monthly history. In addition to usage totals (active client count and its breakdown into entities and non-entity tokens), each tab will include a list of the top ten namespaces by client count and the ability to export client count data for all namespaces. Here is a screenshot for your reference:

Vault Client Count

Q: In versions prior to Vault 1.9, how do I compute changes to clients month to month from the UI?

To perform this calculation, you must know the billing period. For the sake of this example, assume your billing period starts on January 1st and ends on December 31st:

  1. To compute new active unique clients (clients who are new to Vault) for January, you need to use the UI to set the billing start date to January 1st and end date to January 31st:

    -Unique clients for Jan = [Jan1 to Jan31]

  2. To compute new active unique clients for February, you need to do the following:

    -Unique clients for Jan + Feb = [Jan 1 to Feb 28/29]

    -Unique clients for Jan = [Jan1 to Jan31]

    -Unique clients for Feb = (Unique clients for Jan + Feb) - (Unique clients for Jan)

  3. To compute new active unique clients for March, you need to do the following:

    -Unique clients for Jan + Feb + Mar = [Jan 1 to Mar 31]

    -Unique clients for Jan + Feb = [Jan1 to Feb 28/29]

    -Unique clients for Feb = (Unique clients for Jan + Feb + Mar) - (Unique clients for Jan + feb) … and so on.

Q: What if I selected an inaccurate billing period via the UI/API?

If you selected an inaccurate billing period, then the clients reflected in the result will be incorrect. The API/UI considers the start date as a fresh start to re-count unique clients that were counted towards the billing period.

Q: What if I want to skip computation of clients for a period of time during the billing period?

If you break the billing period, duplicates may likely occur. For instance, assume the start date of your billing period is January 1st and December 31st. You compute clients for period 1 = January 1st to March 31st and then period 2 = May 1st to December 31st (thus skipping the month of April). Clients that were already counted during period 1 may be recounted in period 2. Therefore, the addition of clients for period 1 + period 2 will include duplicates.

Q: What are the known client count issues in the auditor tool as well as in the UI/API?

Known issues for both tools include the following:

Auditor tool:

  • New updates to the client count logic (made in the API/UI starting Vault 1.8) are not reflected in the Vault auditor tool. We do not plan to update the auditor tool to be consistent with the API/UI and will eventually deprecate it. Use this tool as a stop-gap until you upgrade to a newer version of Vault

UI/API:

  • Via the UI/API, the billing period cannot be computed for start and end dates that fall in the middle of a month. For example, if the billing period starts on March 15th and ends on March 14th within the subsequent year, the tool can only compute the billing period assuming March 1 is the start date or April 1 is the start date, but not using the March 15th start date
  • As of Vault 1.9, KMIP clients are not provided by the API/CLI. We have plans to add this to a future version

Q: Under what conditions can cause the loss of client data?

The activity log (component within Vault responsible for computing clients) is tracked on standby nodes and periodically transmitted to the active node over gRPC. The transmission is triggered when the information on the standby node reaches a maximum size of 8KB or 10 minutes has elapsed.

Should a Vault node go down any time during the 10-minute window (e.g., at the 8-minute mark of that 10-minute window), any client activity from those 8-minute period, as well as the time period while the Vault node is down, will be lost. This behavior is expected as there are no activities taking place if the node is down. Lost client activity is acceptable partly because it works in the customer’s favor - if a new client has activity during the lost window, the customer would not be billed for that client.

Q: How can I disable the counting of client activity?

You can disable the count of client activity by using the enabled parameter in the client count config. For more information, refer to the documentation on Update the Client Count Configuration. Disabling the feature during the middle of a month will discard any data recorded for that month but does not delete previous months.

Q: If I request data for January 2021 - December 2021, but April’s data does not exist, what will be included in the total client count result?

In this scenario where you requested data for January 2021 through December 2021, and April's data does not exist because tracking was disabled for that particular month while other months have data, Vault will only return the most recent contiguous set of data, so in this case, data will only be returned for May 2021 through December 2021.

Q: How can I configure the activity for log retention?

Specify retention months in the client config by following the steps in the documentation on Update the Client Count Configuration.

Q: Do child namespaces create duplicate tokens?

A token created in a parent namespace can be used in a child namespace without adding additional clients.

However, creating a new token across a parent/child namespace boundary could result in a token without an entity and a new client. This is because identity is scoped to a single namespace and tokens cannot be associated with identities that live outside their namespace.

Q: How does the Nomad Vault integration affect client counts?

The Nomad Vault integration uses token roles. A single token role creates tokens for many Nomad jobs. If no explicit identity aliases are provided (which is not currently supported in the integration), this would create a non-entity token for every running instance of a Nomad job. Prior to Vault 1.9, the Nomad Vault integration caused duplicate clients, resulting in an elevated client count. Post 1.9, with the introduction of the deduplication logic, the number of clients created by the integration is reduced. For more information on improvements made to client count in Vault 1.9, refer to the question Which version of Vault reflects the most accurate count of clients with Vault?.