MTLS with Traefik & Smallstep

6 min readMay 8, 2023

In enterprise world, Mutual TLS, Mutual Authenticated SSL, 2 way TLS, 2 Way SSL all point out to the great concept of mandating requestor to present it’s certificate and key while calling the apis, system, app, etc. The certificate famously known as client certificate, since — the client is supposed to send it along with request.

Most of my techie friends and readers of this blog must have had experience with MTLS related issues quite often(during their initial days, if not very often), and most of the times they must have had some trust issues (😀 — I mean trust-store issues) or the key is not correct, cert expired, etc.

This blog post of mine is dedicated to gaining confidence with the MTLS on your local & explore the overall flow end to end.

Here I will share the details to setting up Traefik & Smallstep on a local machine, to create certificates using ACME protocol, and also setup client certificate(and key) for the integration.

Main Requirements:-

A nice computer, more recent if better.
Some knowledge of Traefik & Smallstep.
least privileges for setting up softwares/ apps.
⚠️ Above average Knowledge of MTLS & TLS.
access to hosts file for creating DNS entries.

Step 1 — Decide on the domain/ DNS

Since every thing will be part of the local machine, to make a domain resolvable. There are two options:-

1 — The easy way add entries into hosts file.

inside the hosts file, locate for the 127.0.0.1 entry, append DNS separated by spaces.

⚠️ Choose your domain names accordingly. ❗️

example:-

127.0.0.1 localhost tfk.neuw server.neuw ca.neuw

The localhost is very important, standard, default entry.
ca.neuw = DNS for the CA, the certificate authority.
tfk.neuw = DNS for the traefik dashboard.
karanbir.neuw = an application running on local to access a POC API.

2 — The hard way, create a DNS server on local — dnsmasq for MAC, coredns for windows (refer google, enough articles are there for this one)

Step 2 — Install Smallstep on local

There are enough details on the installation page of smallstep here — https://smallstep.com/docs/step-cli/installation/index.html

The further steps are below:-

Validate step cli is installed.

% step --version # check/ validate if step is installed properly
Smallstep CLI/0.24.3 (darwin/arm64)
Release Date: 2023-04-15 00:54 UTC

2. Initialise the step ca

change the profile and context based on your requirements.
the command is very interactive, it is based on the inputs, the reference input details are below, but you may fill up things based on your setup.

% step ca init --profile=neuw.ca --context=neuw.ca                                                                  
✔ Deployment Type: Standalone
What would you like to name your new PKI?
✔ (e.g. Smallstep): Neuw
What DNS names or IP addresses will clients use to reach your CA?
✔ (e.g. ca.example.com[,10.1.2.3,etc.]): localhost,ca.neuw
What IP and port will your new CA bind to? (:443 will bind to 0.0.0.0:443)
✔ (e.g. :443 or 127.0.0.1:443): :54321
What would you like to name the CA's first provisioner?
✔ (e.g. you@smallstep.com): karanbir@example.com
Choose a password for your CA keys and first provisioner.
✔ [leave empty and we'll generate one]: 

Generating root certificate... done!
Generating intermediate certificate... done!

✔ Root certificate: <USER_HOME>/.step/authorities/neuw.ca/certs/root_ca.crt
✔ Root private key: <USER_HOME>/.step/authorities/neuw.ca/secrets/root_ca_key
✔ Root fingerprint: df723c687b9b2c7860d793f9ba0baba367d614587569e948518d16ae0ffe8b62
✔ Intermediate certificate: <USER_HOME>/.step/authorities/neuw.ca/certs/intermediate_ca.crt
✔ Intermediate private key: <USER_HOME>/.step/authorities/neuw.ca/secrets/intermediate_ca_key
✔ Database folder: <USER_HOME>/.step/authorities/neuw.ca/db
✔ Default configuration: <USER_HOME>/.step/authorities/neuw.ca/config/defaults.json
✔ Default profile configuration: <USER_HOME>/.step/profiles/neuw.ca/config/defaults.json
✔ Certificate Authority configuration: <USER_HOME>/.step/authorities/neuw.ca/config/ca.json

Your PKI is ready to go. To generate certificates for individual services see 'step help ca'.

3. Add a provisioner for ACME protocol — later required by traefik. Override the properties accordingly, where ever required.

% step ca provisioner add acme --type ACME --x509-min-dur 1h --x509-default-dur 9490h1m0s --x509-max-dur 9490h1m0s
# --x509-min-dur 1h = 1 hour = minimum certificate duration
# --x509-default-dur 9490h1m0s = 13 months = default certificate duration
# --x509-max-dur 9490h1m0s = 13 months = mac certificate duration

4. Install the root certificate on the local machine.

There will be prompts to authorise the request, you will be asked for biometrics or system password.

step certificate install <USER_HOME>/.step/authorities/neuw.ca/certs/root_ca.crt

5. Start smallstep CA server.

step-ca --context=neuw.ca
# replace the context based on your setup/ instruction

If run successfully you will get the below log at the end:-

2023/05/08 10:36:31 Serving HTTPS on :53421 …

Step 3— Install Traefik on local

Go to the https://github.com/traefik/traefik/releases, download the release specific to the machine/ system.

Follow instructions here — https://doc.traefik.io/traefik/getting-started/install-traefik

traefik.yml file:-

accessLog:
  bufferingSize: 100
  filePath: log-access.log # relative to current location, will be created
api:
  dashboard: true
  debug: true
entryPoints:
  web:
    address: ":80"
    forwardedHeaders:
      insecure: true
  websecure:
    address: ":443"
log:
  filePath: log-file.log # relative to current location, will be created
  level: DEBUG
providers:
  file:
    filename: <location>/tfk.yml # replace location as per setup
serversTransport:
  insecureSkipVerify: true
certificatesResolvers:
  local:
    acme:
      caserver: https://ca.neuw:54321/acme/acme/directory
      storage: <location>/acme.json # replace location as per setup
      httpChallenge:
        entryPoint: web

create a file acme.json

For linux/mac — make sure to chmod 600 for the file, it is referred in above traefik.yml

For windows machine please refer — https://stackoverflow.com/questions/5264595/windows-chmod-600

tfk.yml:-

http:
  routers:
    dashboard:
      rule: "Host(`tfk.neuw`)"
      service: "noop@internal"
      entryPoints:
        - "web"
      middlewares:
        - "secured-redirect"
    dashboard-secured:
      entryPoints:
        - "websecure"
      rule: "Host(`tfk.neuw`)"
      service: "api@internal"
      tls:
        certResolver: "local"
        domains:
          - main: "tfk.neuw"

    karan:
      rule: "Host(`karanbir.neuw`)"
      service: "noop@internal"
      entryPoints:
        - "web"
      middlewares:
        - "secured-redirect"
    karan-secured:
      entryPoints:
        - "websecure"
      rule: "Host(`karanbir.neuw`)"
      service: "example-service"
      tls:
        certResolver: "local"
        options: acmeClient
        domains:
          - main: "karanbir.neuw"          

  middlewares:
    secured-redirect:
      redirectscheme:
        scheme: https
        permanent: true

  services:
    example-service:
      loadBalancer:
        passHostHeader: true
        servers:
          - url: "http://localhost:8080/"

tls:
  options:
    acmeClient:
      clientAuth:
        caFiles:
          - <location>/root_ca.crt
# the path of root certificate might be like below:-
# <USER_HOME>/.step/authorities/neuw.ca/certs/root_ca.crt
        clientAuthType: RequireAndVerifyClientCert

log:
  filePath: log-file.log # relative to current location, will be created
  level: DEBUG

Step 4 — Start the traefik server

configfile location can be relative or absolute, do that accordingly.

./traefik --configfile=./traefik.yml

if all good. based on the setup on the specific system and if domain for the traefik dashboard is tfk.neuw — in that case you will be able to access traefik in the browser over https://tfk.neuw or what so ever domain you have configured.

Step 5— A mock app behind traefik proxy.

Create an application of your choice & technology of choice. I used a Java Application that is running on the port 8080.

The example-service in the above tfk.yml configuration is part of this step.

Step 6— Creating certificate for MTLS connection.

Use the below command to create a certificate and it’s corresponding private key, stored at client.crt and client.key file location respectively

step ca certificate client client.crt client.key --set emailAddresses=example@neuw.in --context=local.mac --size=4096 --kty=RSA

create a .p12 — pkcs12 bundle from this:-

step certificate p12 client.p12 client.crt client.key
# enter password when prompted

Step 7 — Test the integration

Negative scenario — Try accessing the url — https://karanbir.neuw (or the domain you have configured in the traefik/ hosts file)in the browser, you may get the below error:-