# Multiple Servers With Load Balancing

> Run several instances behind a load balancer: license each, clone security profiles with the admin endpoints or use authentication servers, set sticky sessions.

- **Product:** DS Server
- **Section:** Documentation
- **Kind:** Topic
- **Path:** DS Server > Installation Guide > Multiple Servers With Load Balancing
- **2089 words**
- **Web URL:** https://docs.textcontrol.com/ds-server/installation/load-balancing/
- **LLMs URL:** https://docs.textcontrol.com/ds-server/installation/load-balancing/llms.txt
- **LLMs-Full URL:** https://docs.textcontrol.com/ds-server/installation/load-balancing/llms-full.txt

---

When one DS Server is not enough, you run several instances behind a load balancer. Each instance is a complete DS Server with its own admin database, so two things need planning: every instance must accept the credentials and access tokens of your applications, and every request of a Document Editor session must reach the instance that holds the session.

What follows sets up a group of servers for administrators and DevOps engineers: licensing each instance, sharing the security profiles by cloning or through authentication servers, and configuring the load balancer.

### Prepare Each Server

1. **Install DS Server** on every server, in the same version, with Docker or any other way from the [Installation Guide](https://docs.textcontrol.com/ds-server/installation/llms-full.txt).
2. **Change the admin password** of each instance. See [Default Admin Password](https://docs.textcontrol.com/ds-server/administration/admin-password/llms-full.txt).
3. **License each instance.** DS Server is licensed per server installation, and a container instance counts as one. Register each instance as described in [Licensing](https://docs.textcontrol.com/ds-server/administration/licensing/llms-full.txt), or with the licensing endpoints of the Web API when you create instances by script. The [Licensing Overview](https://docs.textcontrol.com/ds-server/overview/licensing/llms-full.txt) explains the licensing model.

### How Instances Share Security Profiles

Each instance keeps its security profiles in its own *Admin.db*, and it keeps the access tokens it issues there as well. A client that requests a token from one instance and sends it to another instance with OAuth enabled is refused with 401 Unauthorized, unless the second instance can check the token with the first. DS Server offers two ways to set up a group:

|  | Cloned security profiles | Authentication servers |
|---|---|---|
| Available since | All versions | Version 3.1.0 |
| Where the profiles are | The same profiles, with the same client IDs and secrets, on every instance | On one or more instances that act as authentication servers |
| Tokens | Valid only on the instance that issued them | Accepted by every instance that has the issuing server registered |
| Changing a profile | Change it on every instance | Change it on the authentication server |
| Load balancer | Sends every request of a client to the same instance | Can send token requests and API calls to any instance; Document Editor connections still need sticky sessions |

### Clone the Security Profiles

Create the security profiles on one instance, the master server, in the admin portal or with the Web API. Then copy them to the other instances with the admin endpoints [GET admin/securityprofiles](https://docs.textcontrol.com/ds-server/reference/web-api/administration/securityprofiles/get/llms-full.txt), [GET admin/securityprofiles/{id}](https://docs.textcontrol.com/ds-server/reference/web-api/administration/securityprofiles-id/get/llms-full.txt) and [POST admin/securityprofiles](https://docs.textcontrol.com/ds-server/reference/web-api/administration/securityprofiles/post/llms-full.txt). The admin endpoints take the admin user name and password in a Basic authorization header.

The following Bash script copies every profile of one instance to another. It needs `curl` and `jq`:

```bash
#!/bin/bash
# Copy every security profile from one DS Server to another
SOURCE="http://dsserver1:5000"
TARGET="http://dsserver2:5000"
SOURCE_ADMIN="admin:<admin password of dsserver1>"
TARGET_ADMIN="admin:<admin password of dsserver2>"

for id in $(curl -s --user "$SOURCE_ADMIN" "$SOURCE/admin/securityprofiles" | jq '.[]'); do
  curl -s --user "$SOURCE_ADMIN" "$SOURCE/admin/securityprofiles/$id" \
    | jq '{name, allowClientCredentialsGrantType, expiryDate, clientCredentials}' \
    | curl -s --user "$TARGET_ADMIN" --header "Content-Type: application/json" \
        --data-binary @- "$TARGET/admin/securityprofiles"
  echo
done
```

The script works as follows:

- **`GET admin/securityprofiles`** returns the IDs of all profiles as a JSON array, such as `[1,2]`.
- **`GET admin/securityprofiles/{id}`** returns one profile with its name, the client credentials (client ID, client secret and authorized redirect URIs), the client credentials grant setting and the expiry date.
- **`jq`** keeps the fields that a new profile takes. `allowClientCredentialsGrantType` and `expiryDate` belong at the top level of the body, not inside `clientCredentials`.
- **`POST admin/securityprofiles`** creates the profile on the target with the same client ID and secret, and returns its new ID. A profile whose client ID already exists on the target is refused with 400 Bad Request and the message "A profile with the given client ID already exists.", so the script can run again after you add profiles.

The OAuth setting is not part of a profile: enable OAuth on each instance, in the admin portal as described in [Enable OAuth](https://docs.textcontrol.com/ds-server/administration/oauth/llms-full.txt) or with [POST admin/oauth](https://docs.textcontrol.com/ds-server/reference/web-api/administration/oauth/post/llms-full.txt). A setting changed through the endpoint takes effect after a restart of the instance.

With cloned profiles, every instance issues its own tokens, and a client must send a token to the instance that issued it. Configure the load balancer to keep each client on one instance, as described below.

### Use Authentication Servers

Since version 3.1.0, an instance can ask other DS Server instances whether a request is authorized. You register those instances as *authentication servers*:

- When [POST oauth/token](https://docs.textcontrol.com/ds-server/reference/web-api/oauth/token/post/llms-full.txt) or [GET oauth/authorize](https://docs.textcontrol.com/ds-server/reference/web-api/oauth/authorize/get/llms-full.txt) can't authorize a request with the instance's own profiles, the instance forwards the request to its registered authentication servers.
- When a request carries an access token that the instance didn't issue, and OAuth is enabled on the instance, it asks its authentication servers whether the token is valid.

This allows a *central authentication server*: one instance holds all security profiles, and the other instances have no profiles and register the central one. The processing load stays on the instance that received the request. For redundancy, several or all instances can be authentication servers, so that requests can be authorized when one of them is unavailable.

You register authentication servers on the **Authentication** page of the admin portal with **+ Register**, a name and the URL of the other instance, or with the admin endpoint [POST admin/authenticationservers](https://docs.textcontrol.com/ds-server/reference/web-api/administration/authenticationservers/post/llms-full.txt):

```bash
curl --request POST 'http://dsserver2:5000/admin/authenticationservers' \
    --user 'admin:<admin password of dsserver2>' \
    --header 'Content-Type: application/json' \
    --data-raw '{"name": "Authentication server", "serverUrl": "http://dsserver1:5000"}'
```

![The Authentication page of the DS Server admin portal with the heading Authentication Servers, the button Register and an empty table with the columns Name and Url.](https://docs.textcontrol.com/assets/img/ds-server/portal-authentication.db09be19.webp)

[Authentication Servers](https://docs.textcontrol.com/ds-server/administration/authentication-servers/llms-full.txt) describes the page and the endpoints.

### Configure Sticky Sessions

A Document Editor session lives on the instance that opened it: the editor keeps a WebSocket connection to that instance. The load balancer must send every connection of a client to the same instance.

Line-of-business applications are often deployed behind a load balancer that distributes incoming traffic to healthy virtual machines or cloud services. Most cloud providers, such as Azure, offer a distribution mode called *source IP affinity*, also known as session affinity or client IP affinity. The load balancer uses a 2-tuple (source IP and destination IP) or a 3-tuple (source IP, destination IP and protocol) to choose the server, so connections from the same client always reach the same instance.

![A diagram of layer-4 load balancing with session persistence in Microsoft Azure: a client's connections pass a load balancer that uses a 2-tuple hash of source and destination IP, and all reach the same of three virtual machines in an availability set.](https://docs.textcontrol.com/assets/img/ds-server/load-balancer-ip-affinity.35865883.webp)

With cloned security profiles, source IP affinity also keeps a client's token requests and API calls on the same instance.

### Next Steps

- [Authentication Servers](https://docs.textcontrol.com/ds-server/administration/authentication-servers/llms-full.txt): register other instances to authorize requests.
- [Creating Security Profiles](https://docs.textcontrol.com/ds-server/administration/security-profiles/llms-full.txt): the profiles you share between instances.
- [Installing with Official Image from Docker Hub](https://docs.textcontrol.com/ds-server/installation/docker-hub/llms-full.txt): start instances as containers.
- [Architecture](https://docs.textcontrol.com/ds-server/overview/architecture/#more-than-one-server): how instances, OAuth and sessions fit together.

---

## Nearby

- **Previous:** [Manual Installation](https://docs.textcontrol.com/ds-server/installation/manual/llms-full.txt)
- **Next:** [Configuration](https://docs.textcontrol.com/ds-server/installation/configuration/llms-full.txt)