<?xml version="1.0" encoding="utf-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>edmondburnett.com</title><link>https://edmondburnett.com</link><description>A personal blog on code, infrastructure, and engineering.</description><item><title>Running a 35-billion-parameter AI agent on an 8 GB Nvidia GPU</title><link>https://edmondburnett.com/post/hermes-local-ai-agent</link><description><![CDATA[Setting up, integrating, and benchmarking Hermes, a self-hosted AI agent.]]></description><pubDate>Fri, 2 Oct 2026 18:10:43 +0000</pubDate><content:encoded><![CDATA[<p>Can a decade-old PC with an early-2020s GPU run a genuinely useful AI agent, using a free model?</p>
<p>My curiosity recently returned about the current state of self-hosted LLMs, and what the realistic use cases and
limitations might be on random commodity hardware. The machine I chose for this experiment is an older headless mini-ITX
Arch Linux box I've been using as a media server, which also hosts a few homelab tasks such as Pi-hole DNS, a k3s
Kubernetes node, and a very nerdy private IRC server (which I'm rather excited to write about sometime). It has an Intel
i7-6700K (from 2015), 32 GB of RAM, and an RTX 3060 Ti graphics card with 8 GB of VRAM. And for the last few days I've
been running <a href="https://hermes-agent.nousresearch.com/">Hermes</a>, an open-source AI agent from Nous Research, backed by
Qwen3.6, a 35-billion-parameter model that was a good fit for the available hardware.</p>
<p><figure><a class="image-link" href="/static/images/hermes-local-ai-agent/at-a-glance-crop@2x.png"><img src="/static/images/hermes-local-ai-agent/at-a-glance-crop.png" width="960" height="527" srcset="/static/images/hermes-local-ai-agent/at-a-glance-crop.png 1x, /static/images/hermes-local-ai-agent/at-a-glance-crop@2x.png 2x" alt="Writing speed 30–34 tokens per second, reading speed 300–375 tokens per second, about 80 seconds of warm-up on the first message, and 6.5 minutes for a real multi-step task." /></a></figure></p>
<p>It's a little faster than I expected, and much more performant than my earlier naive attempts at running local LLMs. The
model writes at 30–34 tokens per second, or roughly 22–25 words per second, which is quicker than many people read. The
slowest part is the first message of a conversation. The agent has a long set of built-in instructions (about 24,000
tokens) that it has to read before answering, which takes around 80 seconds. After that it's cached, and follow-up
replies begin much faster.</p>
<p>This model is built from many small specialist sub-networks, and only a few of them are active for each word it
produces. Some tuning was needed to fit the most critical parts onto the GPU's 8 GB of VRAM, while the rest sit in
ordinary system memory.</p>
<p>How much of the model sits on the GPU VRAM vs system RAM makes a measurable difference:</p>
<p><figure><a class="image-link" href="/static/images/hermes-local-ai-agent/gpu-offload-tuning-crop@2x.png"><img src="/static/images/hermes-local-ai-agent/gpu-offload-tuning-crop.png" width="960" height="541" srcset="/static/images/hermes-local-ai-agent/gpu-offload-tuning-crop.png 1x, /static/images/hermes-local-ai-agent/gpu-offload-tuning-crop@2x.png 2x" alt="Writing speed rises from 29.8 to 35.6 tokens per second as fewer expert layers stay in system RAM; at 28 the model no longer fits in 8 GB." /></a></figure></p>
<h2>Use cases</h2>
<p>Hermes isn't just a chatbot. Nous Research describes it as a self-improving agent, amassing memories and a library of
skills over time, and learning from a profile it builds on your interactions. It can run arbitrary commands (sandboxed
or otherwise, depending on how you set it up), write code, browse the web, interface with MCP servers, or act as an MCP
server itself.</p>
<p>One simple task I ran was to fetch my website, pull the colour scheme out of its stylesheet, and generate an image of
the palette. It worked through the problem in 17 steps and finished in about six and a half minutes.</p>
<p><figure><a class="image-link" href="/static/images/hermes-local-ai-agent/where-the-time-went-crop@2x.png"><img src="/static/images/hermes-local-ai-agent/where-the-time-went-crop.png" width="960" height="541" srcset="/static/images/hermes-local-ai-agent/where-the-time-went-crop.png 1x, /static/images/hermes-local-ai-agent/where-the-time-went-crop@2x.png 2x" alt="Where 388 seconds went: 79 seconds of warm-up, 64 seconds reading new information, 205 seconds writing, 40 seconds running tools." /></a></figure></p>
<p>Longer conversations are also reasonably fast. With 32,000 tokens of history, around where the agent starts summarising
older messages to save space, it's only about 10% slower than with an empty conversation.</p>
<p><figure><a class="image-link" href="/static/images/hermes-local-ai-agent/speed-vs-context-length-crop@2x.png"><img src="/static/images/hermes-local-ai-agent/speed-vs-context-length-crop.png" width="960" height="541" srcset="/static/images/hermes-local-ai-agent/speed-vs-context-length-crop.png 1x, /static/images/hermes-local-ai-agent/speed-vs-context-length-crop@2x.png 2x" alt="Writing speed falls from 34.5 to 31.0 tokens per second, and reading speed from 387 to 353, as the conversation grows from empty to 32,000 tokens." /></a></figure></p>
<p>These sorts of benchmarks begin to inform the potential use-cases. It is, of course, slower than cloud-based frontier
agents. My vision for it lies in less-complex duties: task &amp; issue management for my GitHub projects, a conversational
Slack/Discord/IRC assistant, offloading simple tasks from paid models, an MCP frontend to my personal
note-taking/knowledge base system, one of several agents in a <a href="https://github.com/block/buzz">Buzz workspace</a>, or
perhaps running <a href="https://github.com/p-e-w/heretic">abliterated</a> models for red-teaming or legitimate security research.</p>
<p>On Discord, it got some scripting use straight away. My girlfriend asked it to build a Python web crawler that finds
climate-related job listings and emails her a weekly digest. It wrote the script and then improved it over several
rounds of feedback, using a combination of a headless browser and public APIs to harvest data from various job
aggregators.</p>
<p>Around the same time, the server began logging memory-pressure warnings, which led me to a configuration fix (more on
that below). It also insisted it couldn't attach code files in Discord, and declined repeated requests to try. The cause
turned out to be its own memory: during early testing it had saved a note telling itself to put files in a sandbox
folder whose path Hermes couldn't attach from, and it kept following that note over my instructions. Its memory is a
plain markdown file, so the fix was simply to delete the offending lines. Being able to read and edit what the agent
believes about itself is one of the nicer parts of running it. For sharing larger disparate bits of code, I also gave it
access to GitHub for Gists, along with a shared private repo.</p>
<h2>Installation &amp; Setup</h2>
<p>The rest of this post will dive into the technical details, for anyone who wants to reproduce something similar. There
are two main pieces: <code>llama.cpp</code> runs the model, while <code>Hermes</code> talks to it as the custom provider. Both run as systemd
services. I also downloaded the model's vision projector <code>mmproj</code> so that the agent can read images.</p>
<pre><div class="code-label">shell</div><code class="language-shell">sudo pacman -S llama-cpp ggml-cuda python-huggingface-hub
sudo mkdir -p /srv/models
sudo hf download unsloth/Qwen3.6-35B-A3B-GGUF --include '*UD-Q4_K_M*' --local-dir /srv/models
sudo hf download unsloth/Qwen3.6-35B-A3B-GGUF --include 'mmproj-F16.gguf' --local-dir /srv/models
sudo chmod -R a+rX /srv/models
</code></pre>
<p>Installing Hermes is pretty straightforward. <code>hermes model</code> opens a wizard where I set up a Custom Endpoint to
<code>http://127.0.0.1:8080/v1</code>, no key, <code>Chat Completions</code>, context <code>65536</code>, and reasoning set to <code>low</code> by default. I
skipped the <a href="https://portal.nousresearch.com/">Nous Portal</a> model selection for my immediate purposes (those are
optional free and paid cloud models, feel free to explore these if you desire).</p>
<pre><div class="code-label">shell</div><code class="language-shell">curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes model
hermes config set terminal.backend docker
hermes config set terminal.docker_image local/hermes-sandbox:pdf  # we create this later
hermes config set model.supports_vision true
</code></pre>
<h3>Choosing the model</h3>
<p>The model is Qwen3.6-35B-A3B, quantised to 4 bits (Unsloth's <code>UD-Q4_K_M</code>, 20.6 GB). The &quot;A3B&quot; is the important part:
it's a mixture-of-experts model, so although it has 35 billion parameters in total, only about 3 billion are active for
each token. That's what makes it usable on this hardware. A dense 35B model would need to read all of its weights for
every token, which from system RAM would be painfully slow. Here, each token only touches a small slice of the expert
weights, so those can live in RAM while the parts used on every token stay on the GPU.</p>
<h3>Serving it with llama.cpp</h3>
<p>Arch Linux packages <code>llama-server</code> with its own systemd unit, so I overrode its command line with a
<a href="https://wiki.archlinux.org/title/Systemd#Drop-in_files">drop-in</a>, by running <code>sudo systemctl edit llama-server</code> and
pasting the following between the commented lines:</p>
<pre><div class="code-label">ini</div><code class="language-ini">[Service]
ExecStart=
ExecStart=/usr/bin/llama-server -m /srv/models/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf \
  -np 1 -c 65536 -ngl 99 --n-cpu-moe 32 \
  --cache-type-k q8_0 --cache-type-v q8_0 -fa on \
  --jinja --host 127.0.0.1 --port 8080 --alias qwen3.6-35b-a3b \
  --mmproj /srv/models/mmproj-F16.gguf --no-mmproj-offload
Restart=on-failure
RestartSec=5
</code></pre>
<p>Then enable and start with <code>sudo systemctl enable --now llama-server</code> or <code>sudo systemctl restart llama-server</code> if it's
already running.</p>
<p>What the flags do:</p>
<ul>
<li><code>-ngl 99 --n-cpu-moe 32</code> puts every layer on the GPU, then moves the expert weights of 32 layers back to system RAM.
This is the main tuning knob, covered below.</li>
<li><code>-c 65536</code> gives a 64K-token context window. Hermes needs a large one: its system prompt alone is about 24K tokens,
and it starts compressing history at half the window.</li>
<li><code>-np 1</code> runs a single slot. With several parallel slots the context is split between them, and the extra per-slot
state was enough to run out of VRAM on first start. One agent only needs one slot.</li>
<li><code>--cache-type-k q8_0 --cache-type-v q8_0 -fa on</code> stores the KV cache at 8 bits instead of 16, roughly halving its
memory with negligible quality loss. A quantised V cache requires flash attention.</li>
<li><code>--jinja</code> uses the jinja chat template embedded in the model file, which is what makes tool calling work.</li>
<li>The <code>--mmproj</code> line loads the vision projector, so the agent can read images, and keeps it on the CPU to save VRAM.</li>
<li><code>--host 127.0.0.1</code> binds the API to localhost.</li>
</ul>
<h3>Tuning the GPU split</h3>
<p>How many expert layers to keep in RAM was the one setting worth benchmarking. After stopping <code>llama-server</code>, I ran
<code>llama-bench</code> at several <code>-ncmoe</code> values, starting high at <code>40</code> and working down until it no longer fit at <code>28</code>:</p>
<pre><div class="code-label">shell</div><code class="language-shell">llama-bench -m /srv/models/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf \
  -ngl 99 -ncmoe 40 -fa 1 -ctk q8_0 -ctv q8_0 -p 512,4096 -n 128
</code></pre>
<p>40, 36, 32 and 30 ran; 28 ran out of VRAM. Writing speed rose from 29.8 to 35.6 tokens per second as more experts moved
onto the GPU (see the chart earlier in this post). I settled on using <code>-ncmoe 32</code> in the systemd unit rather than the
faster 30, because the benchmark doesn't account for a full 64K context and the vision projector, and a server that
falls over under load is worse than one that's 3% slower.</p>
<h3>The memory-pressure fix</h3>
<p>Early on, I had used <code>--load-mode none</code> to stop llama.cpp memory-mapping the model file, which copies the weights into
the process's own memory instead. On a 32 GB machine with a 20 GB model, that left about 6 GB for everything else, and
the server began swapping and logging memory-pressure warnings.</p>
<p><figure><a class="image-link" href="/static/images/hermes-local-ai-agent/mmap-memory-fix-crop@2x.png"><img src="/static/images/hermes-local-ai-agent/mmap-memory-fix-crop.png" width="960" height="541" srcset="/static/images/hermes-local-ai-agent/mmap-memory-fix-crop.png 1x, /static/images/hermes-local-ai-agent/mmap-memory-fix-crop@2x.png 2x" alt="Removing --load-mode none raised available RAM from 6.1 to 27 GiB and cut model load time from about 60 seconds to 2.6, with no measurable change in writing speed." /></a></figure></p>
<p>Dropping the flag fixed it. With the default <code>mmap</code>, the weights in system RAM are backed by the file on disk and held
in the page cache. They're still in memory, but they're clean pages that the kernel can reclaim and re-read rather than
swap out, so they no longer count against everything else on the box. The 27 GiB &quot;available&quot; figure overstates the real
headroom for that reason: if other programs did claim that memory, the model would slow down while it re-read its
weights from disk. In practice it hasn't done that (yet), and writing speed was unchanged (33.7 tokens per second in
use, against 34.4 benchmarked). The model also loads in 2.6 seconds instead of a minute.</p>
<h3>Hermes</h3>
<p>Hermes connects to <code>llama-server</code> as a custom provider. The setup wizard wrote most of this in <code>~/.hermes/config.yaml</code>.
The <code>context_length</code> setting matters, because it tells Hermes how large the window actually is:</p>
<pre><div class="code-label">yaml</div><code class="language-yaml">model:
    default: &quot;qwen3.6-35b-a3b&quot;
    provider: &quot;custom&quot;
    base_url: &quot;http://127.0.0.1:8080/v1&quot;
    api_mode: chat_completions
    supports_vision: true

custom_providers:
    - name: Local (127.0.0.1:8080)
      base_url: http://127.0.0.1:8080/v1
      model: qwen3.6-35b-a3b
      api_mode: chat_completions
      models:
          qwen3.6-35b-a3b:
              context_length: 65536
</code></pre>
<h3>The Docker sandbox</h3>
<p>Hermes can run commands directly on the host, but for obvious reasons, I want its work to be confined to a container. It
supports a Docker backend, so I built a custom image on top of Nous Research's sandbox, adding tools for documents,
media, scripting and GitHub. I created <code>~/hermes-sandbox/Dockerfile</code>:</p>
<pre><div class="code-label">dockerfile</div><code class="language-dockerfile"># ~/hermes-sandbox/Dockerfile
FROM nousresearch/hermes-sandbox:desktop
ENV PIP_ROOT_USER_ACTION=ignore
RUN apt-get update &amp;&amp; apt-get install -y --no-install-recommends \
      git curl jq ripgrep fd-find file zip unzip xz-utils tree wget fzf less \
      poppler-utils pandoc tesseract-ocr ocrmypdf \
      ffmpeg sox imagemagick sqlite3 \
      build-essential lua5.4 shellcheck \
      dnsutils iputils-ping netcat-openbsd openssl \
 &amp;&amp; rm -rf /var/lib/apt/lists/* \
 &amp;&amp; pip install --no-cache-dir pypdf pandas pyyaml requests beautifulsoup4

# github cli integration
RUN mkdir -p -m 755 /etc/apt/keyrings \
 &amp;&amp; curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg -o /etc/apt/keyrings/githubcli-archive-keyring.gpg \
 &amp;&amp; chmod go+r /etc/apt/keyrings/githubcli-archive-keyring.gpg \
 &amp;&amp; echo &quot;deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main&quot; &gt; /etc/apt/sources.list.d/github-cli.list \
 &amp;&amp; apt-get update &amp;&amp; apt-get install -y --no-install-recommends gh \
 &amp;&amp; rm -rf /var/lib/apt/lists/* \
 &amp;&amp; git config --system user.name &quot;&lt;name&gt;'s Hermes&quot; \
 &amp;&amp; git config --system user.email &quot;&lt;github email address&gt;&quot; \
 &amp;&amp; git config --system credential.https://github.com.helper '!gh auth git-credential'
</code></pre>
<p>This is built by running <code>docker build -t local/hermes-sandbox:pdf ~/hermes-sandbox</code>, and wired up in
<code>~/.hermes/config.yaml</code>:</p>
<pre><div class="code-label">yaml</div><code class="language-yaml">terminal:
    backend: &quot;docker&quot;
    docker_image: local/hermes-sandbox:pdf
    docker_volumes:
        # same path inside and out, so the agent's MEDIA: attachment paths resolve
        - &quot;/home/edmond/.hermes/cache/documents:/home/edmond/.hermes/cache/documents&quot;
    docker_forward_env: [&quot;GH_TOKEN&quot;]
    container_cpu: 1
    container_memory: 5120 # MB
    container_disk: 51200 # MB
    container_persistent: true
</code></pre>
<p>With <code>container_persistent: true</code>, note that changes to docker volumes only take effect after you <code>docker rm -f</code> on the
old container.</p>
<h3>GitHub</h3>
<p>I created a fine-grained access token on GitHub, limiting the agent to Gists and a few select repositories. Hermes
supplies the token into the sandbox as <code>GH_TOKEN</code> (the <code>docker_forward_env</code> line above), where the <code>gh</code> CLI picks it up,
and the git credential helper in the Dockerfile lets <code>git push</code> use it too. Its commits are authored as a separate
identity, so they're easy to tell apart from mine.</p>
<p>Set the token with <code>hermes config set GH_TOKEN &lt;github_token&gt;</code>, which gets stored in <code>~/.hermes/.env</code>.</p>
<h3>Discord integration</h3>
<p>In the <a href="https://discord.com/developers/applications">Discord developer portal</a>, I created a new Discord application. On
the Installation page, set the Install Link to <code>None</code>. On the Bot page, enable <code>Message Content Intent</code> and
<code>Server Members Intent</code>, and turn <code>Public Bot</code> off. Save and hit <code>Reset Token</code> to get the application token. You'll also
need the application ID from the General Information page.</p>
<p>Discord user and channel access is restricted in Hermes's <code>.env</code> with <code>DISCORD_ALLOWED_USERS</code> and
<code>DISCORD_ALLOWED_CHANNELS</code>, and <code>DISCORD_FREE_RESPONSE_CHANNELS</code> lets it reply in some channels without being
@mentioned. In the Discord client, you can right-click on user and channel names to get the IDs, listing them
(comma-separated) in the appropriate environment vars. The application token gets assigned to <code>DISCORD_BOT_TOKEN</code>.</p>
<p>You then invite the bot to your server with
<code>https://discord.com/oauth2/authorize?client_id=&lt;APP_ID&gt;&amp;scope=bot+applications.commands&amp;permissions=309238025280</code>,
replacing <code>&lt;APP_ID&gt;</code> with the one you copied.</p>
<pre><div class="code-label">shell</div><code class="language-shell">hermes gateway install
sudo loginctl enable-linger $USER
hermes gateway restart
journalctl --user -u hermes-gateway -f
</code></pre>
<p>Perhaps it's obvious, but just be aware that when Discord or some other commercial app is the interface, your
conversations pass through their servers. The model runs locally, but you do lose privacy for those sessions. For
anything sensitive, use the terminal interface instead, or perhaps some other secure chat integration (e.g. a private
IRC).</p>
<h3>Terminal access from other machines</h3>
<p>The excellent <a href="/static/images/hermes-local-ai-agent/hermes-tui.png">Hermes agent TUI</a> runs on the server. I use it from
my workstation and laptop over SSH. In <code>~/.ssh/config</code> I added:</p>
<pre><div class="code-label">sshconfig</div><code class="language-sshconfig">Host hermes
    HostName shade
    RequestTTY yes
    RemoteCommand ~/.local/bin/hermes --tui

Host hermes-tmux
    HostName shade
    RequestTTY yes
    RemoteCommand tmux -u new -A -s hermes '~/.local/bin/hermes --tui'

Host hermes-zellij
    HostName shade
    RequestTTY yes
    RemoteCommand zellij attach --create hermes options --default-layout hermes
</code></pre>
<p><code>ssh hermes</code> opens a fresh session, while <code>ssh hermes-tmux</code> and <code>ssh hermes-zellij</code> variants attach to a persistent
session that survives disconnects. tmux needs <code>-u</code> to render the TUI's Unicode characters properly. The
<a href="https://zellij.dev/">zellij</a> variant uses a custom layout that I created at <code>~/.config/zellij/layouts/hermes.kdl</code> on
the server:</p>
<pre><div class="code-label">kdl</div><code class="language-kdl">layout {
    pane command=&quot;zsh&quot; {
        args &quot;-lc&quot; &quot;hermes --tui&quot;
        close_on_exit true
    }
    pane size=1 borderless=true {
        plugin location=&quot;compact-bar&quot;
    }
}
</code></pre>
<h3>Installing a memory provider</h3>
<p>After storing a handful of memories, Hermes might complain about memory size limitations. This is solved by installing
an
<a href="https://hermes-agent.nousresearch.com/docs/user-guide/features/memory/#external-memory-providers">external memory provider</a>.</p>
<pre><code>hermes config set memory.provider holographic
</code></pre>
<p>I chose <a href="https://hermes-agent.nousresearch.com/docs/user-guide/features/memory-providers/#holographic">Holographic</a>
since it's local, has no dependencies, and just uses a regular SQLite database.</p>
<h3>Useful commands</h3>
<pre><div class="code-label">shell</div><code class="language-shell">hermes gateway restart  # after config changes
hermes config set &lt;key&gt; &lt;value&gt;
hermes memory setup  # memory provider wizard
hermes mcp  # manage MCP servers
hermes lsp status

journalctl --user -u hermes-gateway -f
journalctl -u llama-server -f
</code></pre>
<h2>Closing thoughts</h2>
<p>Along with my machine learning studies, having a real system to poke at makes the theory a little more concrete. Running
and tuning my own local model and agent inspires all sorts of fun integration possibilities that will no doubt carry
over to my other work. I also ended up learning more about how to choose a model, mixture-of-experts architectures,
attention variants, and expert routing and quantisation. I'm looking forward to conducting more experiments with
self-hosted models on different types of hardware, or perhaps renting some compute time on a tensor core GPU, such as
the Nvidia A100.</p>
]]></content:encoded></item><item><title>Uxntal Syntax Highlighting Plugin for Neovim</title><link>https://edmondburnett.com/post/uxntal-syntax-plugin-neovim</link><description><![CDATA[Ported a vim syntax highlighter plugin for the Uxntal assembly language to Neovim.]]></description><pubDate>Wed, 5 Aug 2026 09:50:22 +0000</pubDate><content:encoded><![CDATA[<p>Yesterday I published <a href="https://github.com/edmondburnett/uxntal.nvim">uxntal.nvim</a>, a file detection and syntax plugin
for Neovim, written in Lua.</p>
<p><figure><a class="image-link" href="/static/images/uxntal-plugin.png"><img src="/static/images/uxntal-plugin.png" width="1364" height="712" alt="Uxntal plugin screenshot" title="Uxntal displayed in Neovim. Colorscheme is Nightfox, font
is Jetbrains Mono." /></a><figcaption>Uxntal displayed in Neovim. Colorscheme is Nightfox, font
is Jetbrains Mono.</figcaption></figure></p>
<p>I had noticed that links to the original vimscript-based plugin were 404ing, thus breaking updates from my Neovim plugin
manager <a href="https://github.com/folke/lazy.nvim">lazy.nvim</a>. After some searching around with no luck, I checked for an
archived copy of the github repo via archive.org, and found it had been retired by the owner, whose account was no
longer public. I was then able to locate this IP-address-only self-hosted <a href="https://167.235.19.20/uxntal.vim/">repo</a>; it
seems the project has been moved off of Github.</p>
<p>It occurred to me that I should either mirror this repo in case it disappears again, or write my own version
specifically for Neovim. It didn't take long to port the plugin from Vimscript to Lua. For completeness, I also added
additional support for matchpairs (i.e. so that <code>%</code> jumps between pairs), <code>:checkhealth</code>, and vimdocs <code>:h uxntal</code>.</p>
<p>I occasionally enjoy experimenting with esolangs, and the <a href="https://wiki.xxiivv.com/site/uxn.html">uxn</a> virtual machine
offers a fun ecosystem along with an interesting thought experiment: <a href="https://permacomputing.net/">permacomputing</a>; or
building software that stays useful for decades, is maintainable by a single person, small enough to run on old or
scavenged hardware, or on vessels with limited power and compute resources (such as
<a href="https://100r.co/site/pino.html">Pino</a>, sailed by the creators of uxn), with no dependency on any particular platform or
company to stay alive.</p>
<p>A plugin's home disappearing because it lived on someone else's infrastructure could be seen as a fitting little case
study in why that appeal is more than aesthetic. For now, I am hosting my Lua version on
<a href="https://github.com/edmondburnett/uxntal.nvim">github</a> and <a href="https://codeberg.org/edmondburnett/uxntal.nvim">codeberg</a>.</p>
]]></content:encoded></item><item><title>Meta DDoS&apos;d my Mastodon instance</title><link>https://edmondburnett.com/post/meta-ai-crawling-ddos</link><description><![CDATA[Handling a denial-of-service incident caused by Meta AI crawlers.]]></description><pubDate>Mon, 15 Jun 2026 15:13:36 +0000</pubDate><content:encoded><![CDATA[<p>A few days ago I received an alert from my host provider: The inbound traffic rate on one of my servers has been
exceeding alarm thresholds for several hours. Upon SSH'ing into the box, the system alerted that the disk was full.
Something aberrant was clearly happening.</p>
<p>According to <code>htop</code>, several processes were pegged at 100% usage. And as you have likely experienced, when disks fill
up, weird stuff can happen: some common troubleshooting commands fail to run, databases show log writing issues, etc. So
I set about cleaning up a few caches and local temp backups, while also checking and shutting down Postgres, Mastodon,
Elasticsearch, Redis, and a few other processes.</p>
<p>With a little headroom on the disk again, I could run <code>ncdu</code> and see that Mastodon's media cache had grown until it
consumed nearly the entire volume. When an instance encounters a remote post, it fetches and caches the attached media —
avatars, header images, photos, video — so it can render that content locally. Normally this cache stays bounded and
gets pruned on a schedule, but the flood of incoming requests was pulling down remote media far faster than any normal
cleanup routine could manage. Once the disk filled, Postgres could no longer write, Elasticsearch tipped over, and the
rest of the stack followed.</p>
<p>The bot was crawling public, federated content by way of my instance. By default, Mastodon will render public posts,
profiles, and hashtag timelines from any other server it has peered with. The crawler was working through paths like
these (anonymized):</p>
<pre><code>GET /@anonymous@piaille.fr/113980271490580520
GET /@anonymous@pixelfed.art/114144537441489304
GET /api/v1/accounts/109303441654200732/statuses?exclude_replies=true
GET /api/v1/statuses/116469961523621540
GET /api/v1/tags/35mmportrait
GET /api/v1/tags/80sMovies
</code></pre>
<p>Each remote account or status is being proxied and rendered locally, and each render is an opportunity for my server to
go fetch and cache yet more remote media. The hashtag and <code>/api/v1/tags/</code> endpoints are even worse, since a single
popular tag fans out to posts from hundreds of other servers. To a crawler that follows every link it finds, one small
Mastodon instance becomes a doorway into the entire fediverse — and a remarkably efficient way to make someone else's
server do enormous amounts of expensive work on its behalf.</p>
<p>Some time ago, in anticipation of Meta linking their platforms with the Fediverse, many instances (including my own)
became signatories of <a href="https://fedipact.online/">Fedipact</a>, a pledge to broadly defederate Meta from the network. But
this method of harvesting bypasses the need to access content through normal federation.</p>
<p>Part of the fix moving forward was to turn this remote-browsing behavior off entirely. To do this, go to
Preferences-&gt;Administration-&gt;Server Settings-&gt;Discovery and turn the two remote options under the Public Timelines
section to <code>Authenticated Users Only</code>. Unfortunately, leaving this feature on is leaving an open invitation for the next
AI harvester to do exactly the same thing. For a small instance, trading a bit of that openness in exchange for a server
that stays online feels like the obvious call.</p>
<p>Outside of this, the softest approach is to disallow the <code>meta-externalagent</code> user agent in your <code>robots.txt</code>. Meta
<a href="https://developers.facebook.com/docs/sharing/webmasters/web-crawlers#identify-4">claims</a> to honor this, but after a few
hours the requests were still coming. I suspect if you aren't prepared with this opt-out ahead of time, the bot simply
proceeds and they don't bother to keep checking for an updated <code>robots.txt</code>.</p>
<pre><div class="code-label">robots_txt</div><code class="language-robots_txt">User-agent: meta-externalagent
Disallow: /
</code></pre>
<p>The next step was to forbid the bot from accessing the site via the nginx config, by adding it to my list of unwanted
agents and returning a 403 Forbidden.</p>
<pre><code># block unwanted user agents (snipped)
if ($http_user_agent ~* (YandexBot|Yandex|ZyBorg|meta-externalagent)) {
    return 403;
}
</code></pre>
<p>Now, requests from the bot are 403'ing, as expected:</p>
<pre><div class="code-label">nginx_log</div><code class="language-nginx_log">2a03:2880:f806:20:: - - [18/May/2026:21:05:05 +0000] &quot;GET /@anonymous@aus.social HTTP/2.0&quot; 403 146 &quot;-&quot; &quot;meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)&quot;
57.141.6.17 - - [18/May/2026:21:05:12 +0000] &quot;GET /@anonymous@sonomu.club/116166841041339741 HTTP/2.0&quot; 403 146 &quot;-&quot; &quot;meta-externalagent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)&quot;
</code></pre>
<p>But since the bot was reducing the readability of the logs, and causing them to grow to hundreds of megabytes by
continuing to flood nginx with erroring requests, I ended up just blocking the IP ranges being used to query the server.
This of course cleaned up the log spam.</p>
<pre><div class="code-label">iptables</div><code class="language-iptables">-A INPUT -s 57.141.6.0/24 -j DROP
</code></pre>
<pre><div class="code-label">ip6tables</div><code class="language-ip6tables">-A INPUT -s 2a03:2880:f806::/48 -j DROP
</code></pre>
<p>Note that these IP ranges are likely regional and may be different for you. Another consideration is you also may want
to continue to allow <a href="https://developers.facebook.com/docs/sharing/webmasters/web-crawlers">other Meta bots</a> to access
your application. <code>facebookexternalhit</code>, for example, caches preview cards when someone posts a link to your site on a
Meta platform.</p>
<p>I suppose the lesson here is that the old default — leave things open in good faith, let the public web be public — is
largely unsustainable in the present technology landscape. None of the steps I took here were difficult, but closing off
public remote posts quietly subtracted something. The fediverse was merely working as intended: any server could be a
window onto any other. Closing that window protects my little instance, but it also makes the network feel a little less
connected in the public commons.</p>
<p>That same calculation is now playing out everywhere. Faced with predatory harvesting that treats every public endpoint
as free training data, most operators are doing exactly what I did, only at larger scale: gating content behind logins,
throwing up &quot;are you human&quot; interstitials, rate-limiting aggressively, or paywalling. Each of these is individually
reasonable. After all, we never accepted this sort of behaviour from spam bots. Collectively, they're remaking the web
into something far less open than it was intended to be — a place where a great deal of human creative output ends up
behind increasing levels of fortification, because the cost of leaving the door open is now too high.</p>
<p>The web was built on a kind of casual generosity — publish something, and anyone can read it — and that generosity is
precisely what is sadly being strip-mined, unlicensed, at industrial scale.</p>
]]></content:encoded></item><item><title>Custom URL rewriting with Squid</title><link>https://edmondburnett.com/post/squid-custom-url-rewriting</link><description><![CDATA[How to perform arbitrary URL rewriting on a caching Squid proxy with a simple bit of Python.]]></description><pubDate>Sat, 7 Nov 2015 19:09:40 +0000</pubDate><content:encoded><![CDATA[<p>I recently ran into a weird issue where I needed to remove a port number from URLs forwarded from a Squid caching proxy
server. In my specific use case, the port number arrives from some custom handling being done for a client, but this
method can be used to redirect or rewrite URLs matching any criteria you specify.</p>
<p>For this purpose, Squid provides the <a href="http://www.squid-cache.org/Doc/config/url_rewrite_program/">url_rewrite_program</a>
directive. This can point to an executable of any sort, such as a Ruby or shell script. In this case, I'm using Python.</p>
<pre><div class="code-label">python</div><code class="language-python">#!/usr/bin/python

import sys
import re

regex = re.compile(r':8082')

def rewrite_url(line):
    &quot;&quot;&quot; If the input URL contains a port number, strip it out &quot;&quot;&quot;
    url_list = line.split(' ')
    old_url = url_list[0]
    new_url = '\n'
    if regex.search(old_url):
        new_url = re.sub(regex, '', old_url, count=1) + new_url
    return new_url

while True:
    line = sys.stdin.readline().strip()
    new_url = rewrite_url(line)
    sys.stdout.write(new_url)
    sys.stdout.flush()
</code></pre>
<p>View on <a href="https://gist.github.com/edmondburnett/e5d0a487e4a898931548">gist</a>.</p>
<p>Within the <code>while True:</code> block, we receive an input string from Squid via stdin. This string gets sent to our
rewrite_url function, which simply grabs the URL, tries to match our criteria using regex, and if a match is found, we
strip the matching string (in this case, a specific trailing port number) from the URL. The script then simply outputs
the new result on stdout, immediately flushing the stdout buffer.</p>
<p>Now, we just need to make this script executable, then edit our squid.conf and point it to our rewriter, and restart
Squid.</p>
<pre><div class="code-label">squidconf</div><code class="language-squidconf">url_rewrite_program /etc/squid/custom_rewrite.py
url_rewrite_children 12
</code></pre>
<p>Here we also ask Squid to spawn 12 Python processes using the
<a href="http://www.squid-cache.org/Doc/config/url_rewrite_children/">url_rewrite_children</a> directive. I was at one time using
this setup in a production application. Adjust as necessary for your environment.</p>
]]></content:encoded></item><item><title>Vim C++ auto-completion with YCM on Archlinux</title><link>https://edmondburnett.com/post/youcompleteme-archlinux</link><description><![CDATA[Getting the YouCompleteMe code-completion engine working on Arch.]]></description><pubDate>Mon, 20 Oct 2014 05:58:02 +0000</pubDate><content:encoded><![CDATA[<p>I've been doing more C++ development lately, and decided to check out the available code hinting solutions for the
language in Vim. The interwebs quickly suggested the <a href="https://github.com/ycm-core/YouCompleteMe">YouCompleteMe</a> plugin
as the thing to try -- however, I hit a few temporary snags during the installation process. While the official docs
include instructions for Ubuntu, performing the same steps on Archlinux might leave you scanning error logs and Googling
for solutions.</p>
<p>YCM uses clang for native C/C++ code completion, and the plugin includes an installation script which builds &amp; installs
a copy under it's own subdirectory. To avoid some dependency issues (which for me involved a missing libedit.so.2),
we're just going to use Archlinux's clang package from the official repos instead.</p>
<p>If you don't already have clang installed, let's take care of that now:</p>
<pre><div class="code-label">shell</div><code class="language-shell">sudo pacman -Sy clang
</code></pre>
<p>Download and install the plugin itself. I use Vundle, which makes this as a simple as adding a line to your .vimrc:</p>
<pre><div class="code-label">vim</div><code class="language-vim">Plugin 'valloric/YouCompleteMe'
</code></pre>
<p>Open Vim, and update/install the plugin:</p>
<pre><div class="code-label">vim</div><code class="language-vim">:PluginUpdate
</code></pre>
<p>Now exit vim and run YCM's installation script, telling it to use the system clang:</p>
<pre><div class="code-label">shell</div><code class="language-shell">cd ~/.vim/bundle/YouCompleteMe
./install --clang-completer --system-libclang
</code></pre>
<p>YCM should now be ready to use. For C/C++/Obj-C projects, don't forget to set your compiler flags in a
.ycm_extra_conf.py file in the project's home directory (refer the
<a href="https://github.com/ycm-core/YouCompleteMe">official docs</a> for more details on usage).</p>
]]></content:encoded></item><item><title>Release of twitter-text-python 1.0.3</title><link>https://edmondburnett.com/post/twitter-text-python-1.0.3</link><description><![CDATA[I recently took over maintenance of this library and released version 1.0.3.]]></description><pubDate>Thu, 31 Jul 2014 02:35:22 +0000</pubDate><content:encoded><![CDATA[<p>I recently took over maintenance of <a href="https://pypi.python.org/pypi/twitter-text-python/">twitter-text-python</a>, a Python
library which parses the content of Tweets, extracting URLs, usernames, and hashtags; and generates HTML for output and
display. Since the Twitter API returns plain statuses without any markup, you'll probably need to handle this in your
application - something that TTP makes incredibly easy.</p>
<p>To install:</p>
<pre><div class="code-label">shell</div><code class="language-shell">pip install twitter-text-python
</code></pre>
<p>Usage example:</p>
<pre><div class="code-label">python</div><code class="language-python">&gt;&gt;&gt; from ttp import ttp

&gt;&gt;&gt; status = &quot;@burnettedmond, go downstairs and throw a tarp&quot; \
    &quot;over anything that says #OperationHenessey on it.&quot; \
    &quot;https://github.com/edburnett&quot;

&gt;&gt;&gt; p = ttp.Parser()
&gt;&gt;&gt; result = p.parse(status)

&gt;&gt;&gt; result.urls
['https://github.com/edburnett']

&gt;&gt;&gt; result.users
['burnettedmond']

&gt;&gt;&gt; result.tags
['OperationHenessey']

&gt;&gt;&gt; result.html
u'&lt;a href=&quot;https://twitter.com/burnettedmond&quot;&gt;@burnettedmond&lt;/a&gt;, go downstairs and throw a tarp over anything that says &lt;a href=&quot;https://twitter.com/search?q=%23OperationHenessey&quot;&gt;#OperationHenessey&lt;/a&gt; on it. &lt;a href=https://github.com/edburnett&quot;&gt;https://github.com/edburnett&lt;/a&gt;'
</code></pre>
<p>Today I'm releasing v1.0.3, which updates the generated URLs for compatibility with version 1.1 of the Twitter API. Some
other planned updates include Python 3 support, allowing the caller to specify a target for links (such as
<code>target=&quot;_blank&quot;</code>), and a refresh of the documentation.</p>
<p>TTP is released under the MIT license. It was originally authored by <a href="https://github.com/BonsaiDen">Ivo Wetzel</a>, and was
previously maintained by <a href="https://github.com/ianozsvald">Ian Ozsvald</a>.</p>
<p>Check out the <a href="https://github.com/edmondburnett/twitter-text-python">project page</a> on Github.</p>
<p>Update: Since the Twitter public API was effectively killed by the new ownership, and the paywall makes testing and
development impractical, this module was archived will no longer be actively maintained. It should work fine with any
existing API data.</p>
]]></content:encoded></item><item><title>Push-To-Deploy with Git, Python edition</title><link>https://edmondburnett.com/post/python-push-to-deploy</link><description><![CDATA[Example of using a Python script for simple application deployment via Git hooks.]]></description><pubDate>Mon, 28 Jul 2014 04:44:33 +0000</pubDate><content:encoded><![CDATA[<p>I have read and used <a href="http://krisjordan.com/essays/setting-up-push-to-deploy-with-git">Kris Jordan's excellent guide</a> on
implementing git hooks to deploy code to a production server. I highly recommend following his tutorial if you're
interested in using git to deploy your application.</p>
<p>There's only one problem: The post-receive script is written in Ruby.</p>
<p>Okay, I guess that's not actually a problem. The Ruby script works just fine, although I did of course have to install
Ruby first to get it to run. However, for those of us who live in the Python universe, and may not be accustomed to
working with Ruby (or have it installed), I thought it might be handy to have a Python version of this script to work
off of.</p>
<pre><div class="code-label">python</div><code class="language-python">#!/usr/bin/env python
# post-receive hook for git-based deployments

import sys
import os
from subprocess import call

# configuration
deploy_to_path = '/path/to/deploy/directory/'
deploy_branch  = 'main'

def post_receive(from_commit, to_commit, branch_name):
    # Don't deploy if pushed branch != deploy_branch
    if not branch_name.endswith(deploy_branch):
        print('Received branch ' + branch_name
            + ', not deploying.')
        sys.exit()

    # copy files to deploy directory
    call('GIT_WORK_TREE=&quot;' + deploy_to_path
        + '&quot; git checkout -f ' + branch_name, shell=True)
    print('DEPLOY: ' + branch_name + '(' + to_commit
        + ') copied to ' + deploy_to_path)

    # TODO: Deployment Tasks
    # i.e. Run a script, restart daemons, etc


if __name__ == '__main__':
    # get values from STDIN
    fc,tc,bn = sys.stdin.read().split()
    post_receive(fc, tc, bn)
</code></pre>
<p>Just name this script <code>post-receive</code> and place it in your bare remote repository under the <code>hooks</code> subfolder. Set
<code>deploy_to_path</code> and <code>deploy_branch</code> to appropriate values, and insert any additional custom tasks to be run at the end
of the <code>post_receive()</code> function. Everything else in Kris' tutorial still applies.</p>
<p>Be sure to make the script executable:</p>
<pre><div class="code-label">shell</div><code class="language-shell">chmod +x post-receive
</code></pre>
<p>There is a <a href="https://gist.github.com/edmondburnett/40e7db34416fdc734846">gist</a> of this script available.</p>
]]></content:encoded></item><item><title>Obligatory introduction post</title><link>https://edmondburnett.com/post/obligatory-intro-post</link><description><![CDATA[The original intro post written for an earlier iteration of this blog.]]></description><pubDate>Sun, 27 Jul 2014 02:03:44 +0000</pubDate><content:encoded><![CDATA[<p>The first post on a new blog is often the most difficult, if not awkward due to the fact that none of the interesting
stuff that naturally comes later has actually happened yet. Not to mention most readers who encounter this post will be
doing so long after it was written. Introductions really mean very little in the online world, especially when we have
yet to establish the usefulness of a fresh and unpoven personal outlet. So why are we always preparing? Let's just go.</p>
<p>Sometime in the late 80's my father brought home a Commodore PC-20. It had a word processor, Hangman, and 640K of
memory. Things became considerably more interesting when I discovered what a modem was, and started learning the Hayes
command set to log into local dial-up BBS's through a POTS telephone line. For an 8 year old kid, finding this whole
other world out there was incredibly exciting. Me and my friends promptly began tying up phone lines for days on end,
downloading warez, discovering early digital art and music, playing ASCII games, and communicating with total strangers.
There were lots of busy signals in those days.</p>
<p>Programming was another thing that I stumbled upon pretty early. Getting started in coding was a different beast back
then, and tracking down the limited learning resources and documentation required a bit of sleuthing. My Tandy 386sx had
MS-DOS with QBasic (remember GOTO statements?), and I eventually figured out enough to cobble together a crude text game
or two. When a copy of Borland Turbo C fell into my hands, I started writing little shareware utilities, BBS door games,
and other experiments.</p>
<p>Perhaps the most important aspect about all of this was that it represented freedom. Here was this new universe where a
kid could explore, communicate, learn, and experiment without rules, without supervision, without fear. Except for a
friend or two, nobody in my life back then really knew or understood what it was we were doing when we spent so much
time using computers. Most other kids thought it was dumb, even something deserving of mockery - but we didn't care. It
felt like we were taking part in some special enigma that few were privy to.</p>
<p>Fast forward to 2014, and things seem to have changed slightly. Technology is no doubt still incredibly interesting, but
I can't help but think that the limitless freedom and potential it represents is being swiftly torpedoed by certain
corporate &amp; political interests, and societal decision makers who themselves barely manage to understand technical
matters at even the most casual level, let alone respect the driving philosophy that made it happen in the first place.</p>
<h2>Something resembling a Mission Statement</h2>
<p>Over the past year or so I feel like I've been consistently inspired, and accumulating enough ideas for articles, that
I'm ready to jump back into doing some regular technical writing.</p>
<p>The primary topics I intend to cover include programming, Linux, cryptography, security &amp; privacy, the FOSS movement,
digital rights, the software industry, computing history, and related areas. Some other conversations of personal
interest may occasionally make an appearance. This proposed direction will no doubt evolve over time.</p>
<p>I've hammered together a minimalist blogging platform. This is a content-centric blog intentionally chopped down to it's
bare essentials. Along those lines, I'm forgoing a comments section, at least for now, and trying something different by
asking readers to e-mail me their comments instead.</p>
<p>I'm looking forward to this. Thanks for reading.</p>
]]></content:encoded></item></channel></rss>