Multiple Processes

Threads in one process share the critical sections of TX Text Control; separate processes don't. That is why Text Control recommends several processes instead of several threads for batch merges of hundreds or thousands of documents, and for services that create documents in parallel. The TXTextControl.DocumentServer.ParallelMailMerge package provides such processes: it runs MailMerge in a pool of worker processes that your own application starts. What follows shows how to set it up in a console application and in ASP.NET Core, how to configure the workers and how to find their number.

The processes of the Document Editor backend, TXTextControl.Web.Server.Core and its worker process for each editor, are a different subject; Multiple Processes in the Deployment Guide describes them.

Why Processes

TX Text Control is thread safe, but it protects shared resources with critical sections, so threads that use it in one process can wait for each other, as Threading explains. In Text Control's sample of 2019, merging 100 templates took about 45 seconds with normal processing and 14 seconds in parallel processes on a CPU with 16 cores, about three times faster. Text Control's MailMerge benchmark of 2023 measured single merges on one thread and recommends a multi-process approach wherever documents are created in parallel, such as in a Web API.

The blog article True Parallel Processing of Documents using MailMerge builds such processes by hand for Windows Forms on the .NET Framework. For .NET 10, the ParallelMailMerge package does it for you.

Install ParallelMailMerge

TXTextControl.DocumentServer.ParallelMailMerge is published on nuget.org and targets .NET 10. It depends on TXTextControl.TextControl.Core.SDK 34.0.4 or higher, and your application needs a valid TX Text Control license: install the Core package from the Text Control Private NuGet Feed as well.

dotnet add package TXTextControl.DocumentServer.ParallelMailMerge
dotnet add package TXTextControl.TextControl.Core.SDK

The package starts the worker processes from your own application assembly, which also provides the TX Text Control license to them. Public NuGet Packages lists the package details.

Merge Documents in Worker Processes

The following console application merges three invoices in parallel. Each job has a DOCX template and a JSON file with the data of one invoice, and the result is a PDF document. Replace the content of Program.cs:

using TXTextControl;
using TXTextControl.DocumentServer;
using TXTextControl.DocumentServer.Parallel;

// A worker process runs its merge here and exits
if (await ParallelMailMerge.TryRunWorkerAsync(args))
{
    return;
}

var jobs = new[]
{
    new { Template = "invoice-1001.docx", Data = "invoice-1001.json", Output = "invoice-1001.pdf" },
    new { Template = "invoice-1002.docx", Data = "invoice-1002.json", Output = "invoice-1002.pdf" },
    new { Template = "invoice-1003.docx", Data = "invoice-1003.json", Output = "invoice-1003.pdf" }
};

await using var mailMerge = new ParallelMailMerge(new TXWorkerOptions
{
    WorkerCount = 5
});

await Task.WhenAll(jobs.Select(async job =>
{
    var result = await mailMerge.MergeJsonDataAsync(
        await File.ReadAllBytesAsync(job.Template),
        FileFormat.WordprocessingML,
        await File.ReadAllTextAsync(job.Data),
        BinaryStreamType.AdobePDF,
        settings: new MailMergeSettings
        {
            UseTemplateFormat = true
        });

    await File.WriteAllBytesAsync(job.Output, result.Document);

    Console.WriteLine($"{job.Output}: {result.QueueMilliseconds} ms queued, {result.TotalMilliseconds} ms in total");
}));

The code works as follows:

• ParallelMailMerge.TryRunWorkerAsync(args) must be the first code of Program.cs. The package starts the worker processes from your application with special command-line arguments; in such a process, the method runs the worker and returns true, and the process ends. In your application's own process, it returns false, and the code continues.
• ParallelMailMerge starts a pool of worker processes. TXWorkerOptions.WorkerCount sets their number. await using stops the workers and releases their resources when the program ends.
• MergeJsonDataAsync merges a template with JSON data in one of the workers. The template arrives as a byte array with its FileFormat, here WordprocessingML for DOCX, and the result is written in a BinaryStreamType, here AdobePDF.
• Task.WhenAll sends all jobs to the same instance at once, and the pool distributes them across the workers. The benefit of the package appears when several independent jobs are sent this way.
• MailMergeResult.Document contains the merged document. QueueMilliseconds, WorkerProcessMilliseconds and TotalMilliseconds tell you how long the job waited for a worker, how long the worker needed and how long it took in total.

MailMergeSettings applies the settings of MailMerge inside the worker before the merge runs, such as UseTemplateFormat, which gives the merged document the page format of the template, RemoveEmptyFields, RemoveEmptyBlocks and the cultures DataSourceCulture and MergeCulture. The merge itself is the MailMerge you know: MergeJsonData runs in the worker process.

Use ParallelMailMerge in ASP.NET Core

In a web application, register one ParallelMailMerge as a singleton and reuse it for all requests. The worker pool then stays alive, instead of starting new workers for every request. The worker handoff is still the first code of Program.cs:

using TXTextControl;
using TXTextControl.DocumentServer;
using TXTextControl.DocumentServer.Parallel;

if (await ParallelMailMerge.TryRunWorkerAsync(args))
{
    return;
}

var builder = WebApplication.CreateBuilder(args);

// One worker pool for all requests
builder.Services.AddSingleton(_ => new ParallelMailMerge(new TXWorkerOptions
{
    WorkerCount = 7,
    RequestTimeoutSeconds = 120
}));

var app = builder.Build();

app.MapPost("/api/mailmerge/pdf", async (
    MergeRequest request,
    ParallelMailMerge mailMerge,
    CancellationToken cancellationToken) =>
{
    var result = await mailMerge.MergeJsonDataAsync(
        Convert.FromBase64String(request.TemplateBase64),
        FileFormat.WordprocessingML,
        request.MergeJson,
        BinaryStreamType.AdobePDF,
        settings: new MailMergeSettings { UseTemplateFormat = true },
        cancellationToken: cancellationToken);

    return Results.File(result.Document, "application/pdf", "merged.pdf");
});

await app.RunAsync();

public sealed record MergeRequest(string TemplateBase64, string MergeJson);

The endpoint receives the DOCX template as a base64 string and the data as JSON, and returns the PDF document. The cancellation token of the request cancels the job when the client disconnects, whether it's still waiting in the queue or already running. In production, templates usually come from your storage or a database instead of the request, and the endpoint needs authentication, as Authentication describes.

Configure the Workers

TXWorkerOptions controls the worker pool:

The values above are the ones the README of the package shows; Text Control publishes no ranges for the options. Each worker runs your own application assembly, started with the dotnet host unless DotNetHostPath names another one.

Find the Number of Workers

More workers aren't automatically faster. In Text Control's measurement, 50 documents were merged with 1, 2, 5 and 7 workers:

Seven workers were 3.6 times faster than one, but each merge took longer in a worker as more workers ran, because they compete for the same machine. The best number depends on the processor cores, the memory, the templates, the output format and the rest of the application. Start with a few workers, measure the whole workload with the times in MailMergeResult and add workers as long as the throughput grows. The blog article Create 100,000 Documents in Under an Hour with Parallel MailMerge in .NET C# describes the measurement; it doesn't name the hardware.

Where the Worker Processes Run

The worker processes run on the machine of the application that starts them, and they use its processor cores and memory. ParallelMailMerge therefore doesn't replace a separate processing service: in an application that also hosts the WebSocket connections of Document Editors, heavy merging can slow down the editors. With many concurrent users, run the merges in a separate application that scales on its own, as Container Scaling describes.

Next Steps

• Threading: TX Text Control in several threads of one process.
• Container Scaling: size and scale the processing service.
• Performance Best Practices: the measures that make each merge faster.
• MailMerge: what the merge component does.