Mastering CancellationToken in Managed Threads for .NET
By Thiago Da Silva Garcia, Software Developer at Authority Partners
In today’s blog we will unravel the power of .NET’s formidable toolkit for multithreading, asynchronous, and parallel programming. In the realm of application development, mastering these tools emerges as a crucial pathway to elevate performance and streamline costs. Among these, a CancellationToken takes the spotlight, offering a strategic advantage by swiftly freeing resources, slashing latency, and optimizing the overall performance of your application.
The Secret (or is it?) World of CancellationTokens
When developing Web APIs, it’s not unusual to face situations where it’s important to free resources from a transaction as soon as possible. This is especially critical when dealing with high-demand and/or low-latency systems.
Why the trouble?
In the realm of front-end applications, you’ve likely encountered the need for a robust mechanism to cancel requests. Chances are, your front-end application already includes a built-in feature for cancelling requests that are no longer needed, for whatever reason. This becomes particularly evident in implementations like the following typeahead field:
Graphic Description: Type ahead sample, showing browser cancelling requests when the user keeps typing for a more specific search term.
In this example, if the user types a new character while the previous request is still pending, that initial request gets cancelled. The browser disregards any subsequent responses from the server, except for the latest one, which is promptly dispatched with the newly specified filter term, as per the user’s request.
As a result, the browser handles all the intricate details, leaving your front-end code to manage only the successful requests necessary to populate the list.
That’s not all, folks!
Cancelling requests on the client-side is a valuable practice. It reduces data exchange, freeing up both browser and network resources, and simplifies UI implementation by minimizing edge cases. However, what implications does this have on the server side?
Graphic Description: Typeahead example demonstrating browser-initiated request cancellations. In this scenario, cancellation isn’t transmitted to the server, as it doesn’t anticipate a CancellationToken in its execution pipeline. Despite the front-end’s act of ignoring and cancelling the initial request, the server processes it, resulting in the heavier first request being returned after the lighter second one.
Merely cancelling a request in the browser doesn’t directly impact the server or release server resources unless specific measures are taken.
To escalate the situation, take a look at the server logs in the bottom right corner of the image: the first request takes longer to process, causing the server to respond after the second request. If the browser hadn’t ignored the response for the first request, it could have posed a problem. The first request, which is no longer relevant, might have overwritten the response of the second request—the one that truly matters.
To avoid the server processing unnecessary requests and returning undesired results to clients, we must ensure the implementation of measures. This is where the CancellationToken proves invaluable.
The CancellationToken
.NET encompasses a request execution pipeline with built-in middleware designed to manage request parameters and context prior to the execution of the designated method.
Within the HTTP layer, as the framework actively establishes the server/client connection, an object is in place to monitor this connection. When the connection is interrupted, a middleware triggers the Cancel() method from a CancellationToken source object situated in this layer. This object is intricately connected as a Cancellation Token parameter within the pipeline. Whenever this parameter is specified in Web API methods, .NET employs its mechanisms to provide you with that reference, ensuring the propagation of the HTTP connection status through this object.
Great power comes with great responsibility.
With that in mind, we have the ability to enhance the capabilities of our APIs significantly by associating a CancellationToken parameter with public Controller methods.
[HttpGet("Foo")]
public Task<string> Foo(CancellationToken cancellationToken){
var client = new HttpClient();
Console.WriteLine("Calling an external service...");
//Passing through the Cancellation Token: Every service downstream will be notified if they also implement Cancellation Token pattern
return client.GetStringAsync("https://localhost:5001/bar", cancellationToken);
}
Naturally, it remains essential to adhere to the CancellationToken lifecycle and understand the points in your code where fast failure can occur.
By using the methods ThrowIfCancellationRequested(), Cancel() and/or the property IsCancelled , you’ll be able to react whenever the browser or client cancels a request.
CancellationToken Source
It’s important to note that the CancellationToken source serves as the primary origin for all cancellation tokens. In the context of managing tokens within a Web API pipeline, it proves particularly useful for exercising control over cancellations. For example, you might want to implement a timeout pattern and ensure its consistent propagation across all methods inheriting the CancellationToken from the initial request.
Having a source for the cancellation token allows you to specify extra parameters, enabling proactive cancellation.
In the provided example, the linkedToken is responsive to both browser cancellations (derived from the CancellationToken parameter) and a 6-second timeout (specified in the ctSource variable). These two elements are interconnected during the creation of linkedToken, ensuring that any cancellation in any of the three variables is mirrored across the others.
[HandleTimeout]
[HttpGet("Example5")]
public Task<string> TimeoutPropagation(CancellationToken cancellationToken){ //Inherited CT
var client = new HttpClient();
Console.WriteLine("Calling a service with a 6-second enforced timeout...");
var ctSource = new CancellationTokenSource(TimeSpan.FromSeconds(6)); //Timeout CT
var linkedToken = CancellationTokenSource.CreateLinkedTokenSource(cts.Token, cancellationToken); //Linking both CTs
return client.GetStringAsync("https://localhost:5001/foo", linkedToken.Token);
}
Timeout Handling
Once a CancellationToken instance is generated, it will automatically undergo cancellation after the designated time specified in the parameter. This proves to be an effective method for ensuring that the system promptly releases resources and confirms that the external connection was genuinely terminated after the stipulated waiting period.
<Sample stack trace for a socket exception after an external call is cancelled due to a timeout>
The Money Maker (or Rather, Saver)
Example of a typeahead feature demonstrating browser-initiated cancellations, when necessary, with cancellation events being transmitted to the backend service (indicated by the logs in the bottom right corner). If a request is cancelled before reaching the “non-returning” stage, the server actively cancels it and issues a null response.
In this demonstration, the implementation follows these guidelines:
Client Side:
- A request is initiated if the user remains inactive for 300ms.
- Any interaction within that timeframe resets the timer.
- If there’s an interaction after the request but before receiving a response, the request is cancelled, and the debounce timer is reset.
- Otherwise, the response list is promptly filled in upon receiving the server response.
var xhr = new XMLHttpRequest();
var timeout;
function debounce(){
if(!!xhr && xhr.readyState != 4 ){
xhr.abort();
}
if(!!timeout){
clearTimeout(timeout);
}
timeout = setTimeout(()=> test() , 300);
}
Server Side:
- An intentional delay of 1 to 4 seconds is introduced for easier demonstration of request behavior.
- When the “non-returning point” is reached, the server refrains from cancelling the request.
- If a cancellation signal is received before reaching certain checkpoints, the request is promptly cancelled, and a null response is immediately returned, with the operation shocking cancelled exception being logged.
- Otherwise, the request is sent back to the frontend as soon as the data becomes available.
[ApiController]
[Route("[controller]")]
public class TypeAheadController : ControllerBase
{
[HttpGet]
public async Task<IEnumerable<string>> Get(string param, CancellationToken ct)
{
try
{
Console.WriteLine($"Searching for '{param}'...");
//Cancellation point 1
var text = await System.IO.File.ReadAllTextAsync("names.json", ct);
var jsonNames = JsonConvert.DeserializeObject<List<string>>(text);
var items = jsonNames.Where(x => string.IsNullOrEmpty(param) || x.ToLower().StartsWith(param.ToLower()))
.ToList();
var delay = Math.Min(items.Count / 3, 3); //For demo purposes, maximum delay will be 4s
//Cancellation point 2
Console.WriteLine($"Waiting server query {param}... {delay + 1}s");
await Task.Delay(TimeSpan.FromSeconds(1), ct);
ct.ThrowIfCancellationRequested();
//Non-returning point (committed to the transaction)
Console.WriteLine("Non-returning point reached");
await Task.Delay(TimeSpan.FromSeconds(delay), CancellationToken.None);
Console.WriteLine($"Returning {param} to API\n");
return items;
}
catch (OperationCanceledException ex)
{
Console.WriteLine($"Cancelled! {param}\n{ex.Message}");
}
return null;
}
}
Find all the code in this demo at thiagosgarcia/CancellationTokenDemo (github.com).
Handling Operation Cancelled Exceptions
Here’s a recommendation on how to manage any OperationCancelledException.
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]
public class HandleTimeoutAttribute : TypeFilterAttribute
{
public HandleTimeoutAttribute() : base(typeof(HandleTimeoutAttributeImpl))
{}
public HandleTimeoutAttribute(Type t) : base(t)
{}
public class HandleTimeoutAttributeImpl : ExceptionFilterAttribute
{
//Using this TypeFilter pattern, you can inject dependencies on attributes...
private readonly ILogger _logger;
public HandleTimeoutAttributeImpl(ILogger logger)
{
_logger = logger;
}
//...and handle whatever you need.
public override void OnException(ExceptionContext context)
{
_logger.LogInformation("Handling exception...");
if(context.Exception is OperationCanceledException)
context.Result = new ObjectResult(null)
{
StatusCode = 408,
Value = "Request cancelled. Please, try again"
};
}
}
}
Notes
Consider using the CancellationToken, but do so with caution:
- Utilize the CancellationToken for read-only actions that don’t impact data integrity.
- Apply the CancellationToken when your action is genuinely cancellable. Once you reach the ”point of no return,” you must decide whether to roll back your actions or proceed with a CancellationToken. None for irreversible actions.
- Use a CancellationToken to release resources from resource-intensive processes. Establish checkpoints where transactions can be avoided—why respond if there’s no recipient on the other end?
- DO NOT use a CancellationToken if there’s a risk of compromising data integrity.
References
Cancellation in Managed Threads — .NET | Microsoft Learn
CancellationToken Struct (System.Threading) | Microsoft Learn
CancellationTokenSource Class (System.Threading) | Microsoft Learn
Recommended patterns for CancellationToken — Developer Support (microsoft.com)
