Reference Tokens and Introspection

Access tokens can come in two shapes: self-contained and reference.

Self-contained tokens are using a protected, time-limited data structure that contains metadata and claims to communicate the identity of the user or client over the wire. A popular format would be JSON Web Tokens (JWT). The recipient of a self-contained token can validate the token locally by checking the signature, expected issuer name and expected audience or scope.

Reference tokens (sometimes also called opaque tokens) on the other hand are just identifiers for a token stored on the token service. The token service stores the contents of the token in some data store, associates it with an infeasible-to-guess id and passes the id back to the client. The recipient then needs to open a back-channel to the token service, send the token to a validation endpoint, and if valid, retrieves the contents as the response.

A nice feature of reference tokens is that you have much more control over their lifetime. Where a self-contained token is hard to revoke before its expiration time, a reference token only lives as long as it exists in the STS data store. This allows for scenarios like

• revoking the token in an “emergency” case (lost phone, phishing attack etc.)
• invalidate tokens at user logout time or app uninstall

The downside of reference tokens is the needed back-channel communication from resource server to STS.

This might not be possible from a network point of view, and some people also have concerns about the extra round-trips and the load that gets put on the STS. The last two issues can be easily fixed using caching.

I presented this concept of the last years to many of my customers and the preferred architecture is becoming more and more like this:

If the token leaves the company infrastructure (e.g. to a browser or a mobile device), use reference tokens to be in complete control over lifetime. If the token is used internally only, self contained tokens are fine.

I am also mentioning (and demo-ing) reference tokens here starting minute 36.

IdentityServer3 supports the reference token concept since day one. You can set the access token type to either JWT or Reference per client, and the ITokenHandleStore interface takes care of persistence and revocation of reference tokens.

For validating reference tokens we provide a simple endpoint called the access token validation endpoint. This endpoint is e.g. used by our access token validation middleware, which is clever enough to distinguish between self-contained and reference tokens and does the validation either locally or using the endpoint. All of this is completely transparent to the API.

You simply specify the Authority (the base URL of IdentityServer) and the middleware will use that to pull the configuration (keys, issuer name etc) and construct the URL to the validation endpoint:

app.UseIdentityServerBearerTokenAuthentication(new IdentityServerBearerTokenAuthenticationOptions
    {
        Authority = "https://localhost:44333/core",
        RequiredScopes = new[] { "api1" }
    });

The middleware also supports caching and scope validation – check the docs here.

There are also multiple ways to revoke a token – e.g. through the application permission self-service page, the token revocation endpoint, by writing code against the ITokenHandle store interface (e.g. from your user service to clean up tokens during logout) or by simply deleting the token from your data store.

One thing our validation endpoint does not support is authentication – this is a non issue as long as you don’t want to use the reference token mechanism for confidentiality.

Token Introspection
Many token services have a reference token feature, and all of them, like us, invented their own proprietary validation endpoint. A couple of weeks ago RFC 7662 – “OAuth 2.0 Token Introspection”, which defines a standard protocol, has been published.

IdentityServer3 v2.2 as well as the token validation middleware starting with v2.3 have support for it.

The most important difference is that authentication is now required to access the introspection endpoint. Since this endpoint is not accessed by clients, but by resource servers, we hang the credential (aka secret) off the scope definition, e.g.:

var api1Scope = new Scope
{
    Name = "api1",
    Type = ScopeType.Resource,
 
    ScopeSecrets = new List<Secret>
    {
        new Secret("secret".Sha256())
    }
};

For secret parsing and validation we use the same extensible mechanism that we use for client secrets. That means you can use shared secrets, client certificates or anything custom.

This also means that only scopes that are included in the access token can introspect the token. For any other scope, the token will simply look like invalid.

IdentityModel has a client library for the token introspection endpoint which pretty much self explanatory:

var client = new IntrospectionClient(
    "https://localhost:44333/core/connect/introspect",
    "api1",
    "secret");
 
var request = new IntrospectionRequest
{
    Token = accessToken
};
 
var result = client.SendAsync(request).Result;
 
if (result.IsError)
{
    Console.WriteLine(result.Error);
}
else
{
    if (result.IsActive)
    {
        result.Claims.ToList().ForEach(c => Console.WriteLine("{0}: {1}",
            c.Item1, c.Item2));
    }
    else
    {
        Console.WriteLine("token is not active");
    }
}

This client is also used in the validation middleware. Once we see the additional secret configuration, the middleware switches from the old validation endpoint to the new introspection endpoint:

app.UseIdentityServerBearerTokenAuthentication(new IdentityServerBearerTokenAuthenticationOptions
    {
        Authority = "https://localhost:44333/core",
        RequiredScopes = new[] { "api1" },
 
        ClientId = "api1",
        ClientSecret = "secret"
    });

Once you switched to introspection, you can disable the old validation endpoint on the IdentityServerOptions:

var idsrvOptions = new IdentityServerOptions
{
    Factory = factory,
    SigningCertificate = Cert.Load(),
 
    Endpoints = new EndpointOptions
    {
        EnableAccessTokenValidationEndpoint = false
    }
};

Reference tokens are a real problem solver for many situations and the inclusion of the introspection spec and authentication makes this mechanism even more robust and a good basis for future features around access token lifetime management (spoiler alert).

HTH

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.

Image may be NSFW.
Clik here to view.