SRE Simplified : Error Budget concept.

The concept of Error Budget is useful for setting up alerts of different severity. This video explains in extremely simplified manner how it works.

The next issue of SRE Simplified is about Service Level Agreement.

Service Level Agreement is an agreement between service supplier and a customer about what the target quality of the service is and what are the consequences if the promise is broken.

But the challenge here is to eastablish monitoring and alerting.

This video provides very simplified example of an SLA for the company that produces apples.

Document Title

What are the SRE concepts, like Service Level Agreement and Reliability?

The challenge is to explain it in simplified manner, without deep discussions on different monitoring strategies.

In this first video I tried to just to scratch the surface.

WireGuard is a relatively new protocol. It claims to be greatly simplified and faster comparing to alternatives, it uses UDP as primary transport and encapsulates encrypted traffic from IP layer. Let's investigate if it's true!

Some differences help to stand out of other VPN solutions:

• Routing as a part of Peer configuration
• QR code for simplified configuration using camera on mobile devices. iPhone, iPad and other things, you'll love it!
• Site-to-site or client-to-server connections are not very different from the configuration perspective
• Supports DNS as part of configuration for Peer
• Easy to implement routing for all traffic or for only for selected networks

I was able to connect Mikrotik, FrtizBox 6690, macos, iPad and iPhone using WireGuard with little efforts.

When it comes to the topic how Prometheus and Alertmanager work together and how to restrict alert notification to fire only on working hours, typicall solution is to apply workaround to avaluated alert rule expession like this

and ON() (hour() < 19 and hour() > 8) and ON() (day_of_week() > 0 and day_of_week() < 6)

The full example of such rule is the following

  - alert: DummyWorkhoursAlert
    expr: 1 and ON() (hour() < 19 and hour() > 8) and ON() (day_of_week() > 0 and day_of_week() < 6)
    for: 5m
    annotations:
      identifier: "{{ $labels.exporter }}/{{ $labels.queue }}"
      summary: "Buyerportal: Dummy alert for working hours {{ $labels.queue }}"
      description: "* don't do anything, dummy alert *"

But the proper solution is to apply mute_time_intervals, as per example below

I use org-mode extensively with org-roam for keeping my knowledge and journals together and working as my brain dump destination. This blog serves kinda the same purpose, so why not to combine these things under Emacs umbrella?

Blogging from Emacs boils down to two options.

TLDR; Hugo can render org file natively, but support of features is quite limited, or use org as source that rendered to markdown. The latter brings issues of content syncing from origin to end result.

In case ruby can’t find OpenSSL 3 headers, use this way.

brew install ruby-install
ruby-install ruby 3.1.4 -- --with-openssl-dir=$(brew --prefix openssl)

How to provision CloudFlare tunnel using Terraform

CloudFlare Tunnel can be useful to use reliable alternative to ngrok when you need to expose your application running locally to the outside world.

The following example exposes my application locally running on port 3000 to the Internet on the hostname https://app.prokopov.me

Prerequisites

• Terraform CloudFlare module v4
• CloudFlare API key with Account:Tunnel permissions.
• Locally installed CloudFlare CLI utility cloudflared https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/install-and-setup/installation/
• Application running locally on port 3000

How does it work?

1. cloudflared CLI is an agent running locally and connected to CloudFlare cloud.
2. DNS record of type CNAME is created, pointing to the CloudFlare cloud.
3. CloudFlare does the routing magic!

Terraform part

resource "cloudflare_tunnel" "main" {
  account_id = "777414c2d4e87234087ebac4685e7df6"
  name       = "tunnel-to-app"
  secret     = random_id.main.b64_std
}

resource "cloudflare_tunnel_config" "main" {
  account_id = "777414c2d4e87234087ebac4685e7df6"
  tunnel_id  = cloudflare_tunnel.main.id

  config {
    warp_routing {
      enabled = true
    }
    ingress_rule {
      hostname = "app.prokopov.me"
      service  = "http://localhost:3000"
    }
    ingress_rule {
      service = "http_status:404"
    }
  }
}

resource "cloudflare_record" "main" {
  value   = "${cloudflare_tunnel.main.id}.cfargotunnel.com"
  proxied = true
  name    = "app"
  type    = "CNAME"
  zone_id = cloudflare_zone.main.id
}

Local tunnel part

1. Find generated token for resource cloudflare_tunnel.main

TOKEN=$(terraform show -json | jq -r '.values.root_module.resources[] | select(.address=="cloudflare_tunnel.main").values.tunnel_token')

2. Use token

cloudflared tunnel run --token=${TOKEN} tunnel-to-app

It is in general very good idea to manage infra configuration as a code. Unfortunately, Mirkotik terrafrom support is basic, as OSS driven.

Nevertheless, I appreciate author for effors.

Here is an example how to use it with Hashicorp Vault.

Commands

export VAULT_ADDR=http://vault_address:8200
export VAULT_TOKEN=<token>
terraform init
terraform plan

Example

main.tf

provider "vault" {}

data "vault_generic_secret" "main" {
  path = "common/mikrotik/nexus-home"
}

provider "mikrotik" {
  host           = data.vault_generic_secret.main.data["address"]  # Or set MIKROTIK_HOST environment variable
  username       = data.vault_generic_secret.main.data["username"] # Or set MIKROTIK_USER environment variable
  password       = data.vault_generic_secret.main.data["password"] # Or set MIKROTIK_PASSWORD environment variable
  tls            = false                                           # Or set MIKROTIK_TLS environment variable
  ca_certificate = "/path/to/ca/certificate.pem"                   # Or set MIKROTIK_CA_CERTIFICATE environment variable
  insecure       = true                                            # Or set MIKROTIK_INSECURE environment variable
}

// /ip address
// :put [find where address="192.168.88.1/24"]
// *1

// terraform import mikrotik_ip_address.lan '*1'
resource "mikrotik_ip_address" "lan" {
  address   = "192.168.88.1/24"
  comment   = "LAN Network"
  interface = "ether2"
}

// uncomment on release https://github.com/ddelnano/terraform-provider-mikrotik
# resource "mikrotik_firewall_filter_rule" "https" {
#   action             = "accept"
#   chain              = "forward"
#   comment            = "Web access to local HTTP server"
#   connection_state   = ["new"]
#   dst_port           = "443"
#   in_interface       = "ether1"
#   in_interface_list  = "local_lan"
#   out_interface_list = "ether3"
#   protocol           = "tcp"
# }

terraform.tf

There are lots of options to greet collegue or close ones with cards.

This one was recommended by one of the coworkers.

Group Greeting Cards