MC Tech Solutions

Tag: monitoring

  • Scrutiny: catching failing hard drives before they die

    Hard drives rarely die without warning: they usually report trouble through S.M.A.R.T. long before they fail. The problem is that nobody reads those reports. Scrutiny reads them for me, and shows the health of every drive in my server on one simple web page.

    What is S.M.A.R.T.?

    Every modern hard drive and SSD keeps self-monitoring data: reallocated sectors, read errors, temperature, power-on hours, wear levels and more. Some of these numbers are strong predictors of failure. Scrutiny collects them on a schedule, stores the history, and flags drives as passed, warning or failed. Its thresholds are based on real-world failure rates published by Backblaze, rather than just the manufacturer’s own limits.

    Setting it up

    I use the “omnibus” image, which bundles the collector, web UI and database in one container. Here’s a compose file based on mine:

    services:
  scrutiny:
    container_name: scrutiny
    image: ghcr.io/analogj/scrutiny:master-omnibus
    restart: unless-stopped
    cap_add:
      - SYS_RAWIO
      - SYS_ADMIN
    ports:
      - "8085:8080"
    volumes:
      - /run/udev:/run/udev:ro
      - ./config:/opt/scrutiny/config
      - ./influxdb:/opt/scrutiny/influxdb
    devices:
      - /dev/sda
      - /dev/sdb
      - /dev/nvme0
    • devices lists every drive to monitor. Find yours with lsblk, and include all of them, especially the big data and backup drives you care about most.
    • SYS_RAWIO lets Scrutiny read S.M.A.R.T. data from SATA drives. SYS_ADMIN is needed for NVMe drives.
    • /run/udev lets it identify drives properly.
    • ./influxdb stores the history, so you can watch trends over time.

    Browse to http://<server-ip>:8085 and you’ll see a card for each drive.

    Reading the dashboard

    • Passed drives need nothing.
    • Warning means an attribute has crossed a threshold that’s linked to higher failure rates. Check that the drive is in your backups and keep an eye on it.
    • Failed means plan a replacement now, and make sure the data is backed up before anything else.

    Click a drive to see every attribute and its history. The ones I watch most are reallocated sector count, pending sectors and uncorrectable errors on hard drives, and “percentage used” on SSDs and NVMe drives.

    Notifications

    A dashboard you never open doesn’t help much. Scrutiny can send alerts through dozens of services (Discord, email, Telegram, ntfy and more) using notification URLs in config/scrutiny.yaml. Set one up so a failing drive actually tells you.

    Tips

    • USB enclosures don’t always pass S.M.A.R.T. data through. If an external drive shows no data, check Scrutiny’s docs for the right device type.
    • Scrutiny is not a backup. It gives you time to act, but drives can still fail suddenly. See my backup plan.

  • My home lab dashboard: Homarr and Glances

    Once you’re running more than a handful of self-hosted apps, remembering which port each one lives on gets old fast. Two tools fix that for me: Homarr, a start page with a tile for every service, and Glances, which shows how the server itself is coping.

    Diagram: a browser on the home network opens the Homarr dashboard, which has tiles for Immich, Nextcloud, Vaultwarden, Plex, Portainer and Technitium, and links to Glances, which reads CPU, memory, disk and Docker stats from the home server.
    Homarr gets me to every service; Glances shows the health of the server underneath.

    Homarr: the front door to everything

    Homarr is a self-hosted dashboard. You build it in the browser by dragging tiles around, with no config files to edit. Each tile links to one of your services and can show whether it’s up. Widgets can add extras like the weather, a calendar or stats from your apps.

    I set it as the home page on my browsers. Photos, files, passwords, Portainer, DNS: everything is one click away, and a red status dot tells me straight away if something has fallen over.

    Setting up Homarr

    Homarr needs a secret key to encrypt the credentials it stores for integrations. Generate one and put it in a .env file next to the compose file:

    mkdir -p /opt/stacks/homarr && cd /opt/stacks/homarr
echo "SECRET_ENCRYPTION_KEY=$(openssl rand -hex 32)" > .env

    Then compose.yaml:

    services:
  homarr:
    container_name: homarr
    image: ghcr.io/homarr-labs/homarr:latest
    restart: unless-stopped
    ports:
      - "7575:7575"
    environment:
      - SECRET_ENCRYPTION_KEY=${SECRET_ENCRYPTION_KEY}
    volumes:
      - ./appdata:/appdata
      - /var/run/docker.sock:/var/run/docker.sock   # optional: lets Homarr see your containers
    docker compose up -d

    Open http://<your-server-ip>:7575, create your admin account, and start adding tiles. The Docker socket mount is optional. It lets Homarr list your containers and add them to the board quickly, but it also gives Homarr a lot of power over your server, so leave it out if you don’t need that.

    Keep it on your home network

    A dashboard is basically a map of your network, so I don’t expose mine to the internet. It goes through Nginx Proxy Manager with an access list that only allows local addresses, so I get a friendly name on my home network and it’s invisible from outside.

    Glances: is the server OK?

    Glances is a system monitor. Think top or htop, but showing everything at once: CPU, load, memory, network, disk activity, free space on every drive, temperatures, the busiest processes and every Docker container’s CPU and memory use. Run it in web mode and you get all of that in a browser tab.

    It’s my first stop when something feels slow. Is it Immich chewing through a big import? Is a drive filling up? Glances answers that in seconds.

    Setting up Glances

    This is my /opt/stacks/glances/compose.yaml:

    services:
  glances:
    container_name: glances
    image: nicolargo/glances:latest-full
    restart: unless-stopped
    pid: host
    network_mode: host
    environment:
      - GLANCES_OPT=-w -C /glances/conf/glances.conf
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /etc/os-release:/etc/os-release:ro
      - ./glances.conf:/glances/conf/glances.conf:ro
      - /media/ExtStorage:/media/ExtStorage:ro

    A few lines are worth explaining, because they’re what let a container see the whole machine:

    • pid: host shows the server’s real processes, not just the container’s.
    • network_mode: host shows real network traffic, and serves the web UI on port 61208.
    • The read-only docker.sock adds per-container stats.
    • /etc/os-release shows the host’s OS name instead of the container’s.
    • Mount any extra drives you want to track (read-only), like my external storage drive.
    • -w starts web server mode.

    Running in a container, Glances also sees a lot of internal mounts that clutter the disk list. A small glances.conf hides them:

    [fs]
hide=/etc/.*,/var/run/.*,/run/.*,/glances/.*

    Start it with docker compose up -d and browse to http://<your-server-ip>:61208.

    Security note: the Glances web UI has no login by default, and it shows a lot about your machine. Keep it on your home network, or start it with --password to require one.

    Putting them together

    Add a Glances tile to your Homarr board, and you have one page that gets you to every service and to the health of the server underneath. With more than one server, run the same Glances compose file on each and give each its own tile.

    New to Docker? Start with Getting started with Docker and Portainer.