The main configuration file for authentication is
sg_authc.yml. It defines how Search Guard retrieves the user credentials, how it verifies these credentials, and how additional user information is fetched from backend systems (optional).
The authentication domains configured in
sg_authc.yml are used for authenticating REST requests to Elasticsearch and for password-based authentication in Kibana. Authentication for the transport client is configured in the additional configuration file
sg_authc_transport.yml. Advanced authentication for Kibana like OIDC or SAML can be configured in
sg_authc.yml file may look like this:
auth_domains: - type: basic/internal_users_db
This way, you configure Search Guard to expect credentials by Basic HTTP authentication. This means, you need to supply a username and password to authenticate. The credentials are checked by Search Guard using its internal users database. If the password is correct, Search Guard allows the user to access Elasticsearch. The user will have the roles that are configured in the internal users database.
type attribute defines a two-dimensional value: The part in front of the slash -
basic in the example - identifies the HTTP authentication frontend. It is responsible for retrieving credentials from the request. The part after the slash is the authentication backend. It is responsible for validating the extracted credentials using a backend system.
Note: The authentication backend is optional. In some cases - like signature-based credentials - the authentication frontend is sufficient to securely authenticate a user. Then, you can just omit the second part of the
A more evolved
sg_authc.yml file may look like this:
auth_domains: - type: basic/internal_users_db - type: basic/ldap ldap.idp.hosts: ldap.example.com ldap.idp.bind_dn: "cn=admin,dc=example,dc=com" ldap.idp.password: secret - type: jwt jwt.jwks_endpoint.url: "https://idp.example.com/public-keys.jwks"
This configuration contains three authentication domains. When Search Guard receives a request, it goes through the list of authentication domains - from top to the bottom - and checks for each authentication domain if it can authenticate the request. When a domain successfully authenticates a request, this process ends. The request passes without checking the succeeding authentication domains.
We already know the first authentication domain of type
basic/internal_users_db. The second one,
basic/ldap also uses Basic HTTP authentication to retrieve the supplied user name and password. However, it connects to an external LDAP server to check the credentials. Here, you can see some more configuration properties that specify how to connect to the external LDAP server. See the LDAP documentation for more details on this.
The third authentication domain has type
jwt. This type only consists of one part, the authentication frontend. The second part, the authentication backend, is not specified here. This means that the authentication frontend is sufficient to authenticate the request. This is because cryptographic signatures are used to verify the authenticity of a user with JWT. When Search Guard is initialized, it loads the public keys for verification as JWKS file from the URL specified using the
jwt.jwks_endpoint.url property. However, the necessary keys can be also specified directly inside the configuration. See the JWT documentation for more on this.
Debugging the authc configuration
If something regarding authentication does not work as expected, you might want to get additional debugging information from Search Guard. Search Guard offers a special authc debugging mode for this.
Note: This mode might reveal sensitive data. Only use this mode on test clusters. Do not forget to switch the debug mode off before going into production.
For this, just add
debug: true to the top level of
debug: true auth_domains: - type: basic/internal_users_db - type: jwt jwt.jwks_endpoint.url: "https://idp.example.com/public-keys.jwks"
You can also use the
sgctl command to set the flag directly on the cluster without having edit files:
$ ./sgctl.sh set authc debug --true
To disable it, use this command:
$ ./sgctl.sh set authc debug --false
If the debug mode is active, open the URL
/_searchguard/auth/debug - either with your browser, curl or your favorite REST client tool. Provide the credentials you want to test. The endpoints returns a JSON document detailing all intermediate steps of the authentication process and their status.
Mapping user information
Both authentication frontends and authentication backends can deliver user information in a big variety of different data structures. Most frontends and backends have suitable defaults for retrieving user names. However, for retrieving roles and user attributes, you often need to specify how you want to retrieve these.
This is where the user mapping comes into play. It allows you to map the various information from authentication frontends and backends to the user name, to roles, and to user attributes.
Let’s have a look at an example:
auth_domains: - type: jwt jwt.jwks_endpoint.url: "https://idp.example.com/public-keys.jwks" user_mapping.user_name.from: jwt.preferred_user_name user_mapping.roles.from: jwt.roles user_mapping.attributes.from: dept_no: jwt.department.number level: jwt.department.access_level
The JWT authentication frontend retrieves the user name by default from the
sub attribute inside the JWT. However, sometimes, this value might be a not really human-readable value like a UUID. Then a different attribute of the JWT might contain a more suitable user name. By specifying
user_mapping.user_name.from: jwt.preferred_user_name we say that the user name should be taken from the
preferred_user_name attribute of the JWT. The expression that can be used as value for this setting is a JSON path, which allows you great flexibility in traversing object graphs for values. In front of
preferred_user_name we need to specify
jwt to indicate that the value shall be taken from the JWT used for authentication. All authentication frontends and backends can make their own information objects available for querying. See the respective documentation of the frontends and backends for the available objects. Also, there are a couple of global objects which provide request metadata such as originating IP addresses, etc.
Moving on to
user_mapping.roles.from: This specifies to retrieve roles from an attribute called
roles inside the JWT. In contrast to the user name, roles are usually multi-valued. Thus, this expects the
roles attribute in the JWT to be an array of role names which will be then mapped to the user’s backend roles.
Often, it might be the case that roles are specified only as a comma-separated string. In this case, you can write instead
user_mapping.attributes.from is the most complex setting. In Search Guard, user attributes are used for dynamically defining the data a user has access to - based on index names or document attribute values. A Search Guard user can have an arbitrary number of named attributes. Each attribute can be a string, number, boolean or a complex type such as an array or an object with further attributes. However, it is not desirable to map all possible attributes coming from the request to user attributes. This is both for security and for performance reasons. Thus, the user attribute mapping should just map the attributes that are really necessary.
In the example, the attribute
department.number is retrieved from the JWT and put into the Search Guard user with the attribute name
dept_no. Additionally, the attribute
level is retrieved from
Using static information for user mapping
You can also define constant values to be used for user name, roles or attributes. For this, use the properties
user_mapping.attributes.static. A case, where this is very useful, is anonymous authentication. See the following example configuration for this:
auth_domains: - type: anonymous user_mapping.user_name.static: anon user_mapping.roles.static: anon_role user_mapping.attributes.static: dept_no: 0 level: 0
Getting the user name from a backend
Usually, the user name is determined by the authentication frontend. For username/password-based authentication, it is quite clear that the username used by Search Guard is normally the same as the one provided by the user during login. Also, JWT, OIDC or SAML protocols usually provide a suitable user name.
In some cases, however, it might be useful to use a backend to provide the actual username which shall be used by Search Guard. For this use-case, you need to use the
An example for this might look like this:
auth_domains: - type: basic/ldap ldap.idp.hosts: - "ldaps://ldap.example.com" ldap.user_search.filter.by_attribute: uid user_mapping.user_name.from_backend: ldap_user_entry.displayName
In this case, the user needs to provide their
uid as user name in the login form. However, when the user is logged in, the
displayName attribute from the corresponding LDAP entry is used as the actual user name.
More details on user mapping
See the section advanced user mapping for a comprehensive overview over all options.
Restricting authentication domains
You can selectively restrict each authentication domains to specific origin IPs or to specific users. This may be useful if you want to offer some authentication domains only for specific systems in your network. This may look like this:
auth_domains: - type: jwt jwt.jwks_endpoint.url: "https://idp.example.com/public-keys.jwks" - type: basic/internal_users_db accept.ips: '10.10.22.16/30' accept.users: 'kibanaserver'
In the example above, JWT authentication is available for all systems. The password-based authentication, however, is only available for certain IPs identified by a CIDR expression. Furthermore, only the user with the name
kibanaserver is allowed to be authenticated on this domain. For the
accept.users property, you can also use wildcards like
accept.users: 'kibana*'. For both properties, you can also specify multiple values as a list.
On the other hand, you can use the
skip.users properties to skip an authentication domain for certain IPs or users:
auth_domains: - type: basic/ldap ldap.idp.hosts: ldap.example.com ldap.idp.bind_dn: "cn=admin,dc=example,dc=com" ldap.idp.password: secret skip.users: 'kibanaserver' - type: basic/internal_users_db accept.users: 'kibanaserver'
You can also combine
accept. When doing so, keep in mind that
accept is evaluated first;
skip is evaluated afterwards. This means that
skip can only further restrict the
accept settings. However,
accept cannot be used to allow access which would be otherwise denied by a
IP addresses of users behind proxies
If users are required to use proxies to access Elasticsearch, their actual IP address might be obscured by the proxy. However, proxies can report the actual IP address of the user in special HTTP headers, such as the
X-Forwarded-For header. Search Guard allows you to evaluate these IP addresses as well. You just need to specify the IP addresses of trusted proxies in the configuration. This is one in a special section of
auth_domains: - type: basic/internal_users_db accept.originating_ips: '172.16.10.0/24' network: trusted_proxies: '10.111.111.0/30'
The originating IP which is checked in
accept.originating_ips is determined the following way:
- If the IP address of the directly connecting host does not match the
trusted_proxiesexpression, that IP is also the originating IP.
- If the IP address is in a trusted proxy network, Search Guard will look at the
X-Forwarded-Forheader. Search Guard will search the IPs specified in the header from right to left for the first untrusted IP address. This will be the originating IP address. If you want to use a different header name, you can configure the name with the property
Additional user information backends
For each authentication domain, you can configure additional user information backends. These user information backends allow you to enrich the user information by additional roles and attributes. The different to normal authentication backends is that user information backends do not participate in authentication: If a user information backend has no records on a particular user, the user is able to login anyway.
Note: Users of earlier versions of Search Guard know “Authorization Backends” as a similar concept. User information backends are replacing these now. Major differences to authorization backends include: While authorization backends were configured globally, user information backends do always belong to one auth domain. This allows for a more fine-grained configuration. Additionally, user information backends support the retrieval of user attributes.
See the following example:
auth_domains: - type: basic/internal_users_dbs additional_user_information: - type: ldap ldap.idp.hosts: ldap.example.com ldap.idp.bind_dn: 'cn=admin,dc=example,dc=com' ldap.idp.password: secret ldap.group_search.base_dn: 'ou=groups,dc=example,dc=com' ldap.group_search.recursive.enabled: true
Here, we use LDAP to retrieve additional roles for users which have authenticated using the internal users database. By default, the roles are represented by the distinguished name of the respective LDAP entries found during the role search. If you want to use other attributes for defining the role names or if you also want to retrieve user attributes, you must use the
The LDAP group search produces an array of LDAP entries, which are made available for user mapping as
ldap_group_entries; each entry contains has a name (
dn) and may have further attributes. Keep in mind that in LDAP attributes are always multi-valued. Thus, the data structure available for user mapping might look similar to this object tree (rendered in YAML):
ldap_user_entry: dn: "cn=Karlotta,ou=people,o=TEST" cn: ["Karlotta"] departmentNumber: [1, 2] ldap_group_entries: - dn: "cn=group1,ou=groups,o=TEST" cn: ["group1"] businessCategory: ["c"] - dn: "cn=group2,ou=groups,o=TEST" cn: ["group2"] businessCategory: ["d", "e"] - dn: "cn=group3,ou=groups,o=TEST" cn: ["group3"] businessCategory: ["f"]
If you want to use the
cn attributes of the group entries as role names and want to map the
businessCategory attributes to user attributes, you can use this user mapping:
auth_domains: - type: basic/internal_users_dbs additional_user_information: - type: ldap [...] user_mapping.groups.from: ldap_group_entries[*].cn user_mapping.attributes.from: biz_cat: ldap_group_entries[*].businessCategory
Remember that the expressions used for retrieving the role and attribute values are JSON path expressions. By writing
ldap_group_entries you specify, that
ldap_group_entries is an array and that you want to search all elements of the array. From all the elements, you are then getting the union of the values of the
There are a number of additional advanced options in
sg_authc.yml. These allow you to customize the following things: