By UNST33R
Inference-time abliteration: steering refusals away without touching weights
A llama.cpp fork that abliterates any model at load time sparked fresh community discussion. How runtime refusal steering compares to baked-in weight edits for red-team harnesses.
Weight-level abliteration requires a per-checkpoint edit: collect contrastive activations, orthogonalize the weight matrices, ship a new model file. Last week an r/LocalLLaMA thread (Inference time abliteration, October 21, 2026) surfaced the other branch of the family tree: doing the same refusal-direction math live, in the inference engine, with zero weight changes.
The project linked in the thread, AbliteratedOutTheBox, is a llama.cpp fork that abliterates whatever model you load, at inference time, with no fine-tuning and no extra files. The thread also ran in r/LocalLLM and r/KoboldAI, with operators comparing notes on latency and behavior drift.
The same idea, applied at a different layer
Both approaches descend from the Arditi et al. finding that refusal is mediated by a single direction in the residual stream (arXiv:2406.11717). The difference is where you subtract it:
- Weight-level: project the refusal direction out of the weight matrices once. The result is a standard checkpoint that runs anywhere.
- Inference-time: keep the original weights and subtract the direction from live activations on every forward pass, inside a patched engine.
Engineering tradeoffs
| Weight-level edit | Inference-time steering | |
|---|---|---|
| New model files | One per checkpoint | None |
| Runtime cost | Zero (projection is baked in) | Activation math every token |
| Portability | Any inference stack | Only the patched engine |
| Per-model tuning | Activation collection pass per base model | Direction estimation still needed per model |
| Ecosystem | Standard GGUF / safetensors workflow | Fork-specific; updates lag upstream llama.cpp |
For a harness operator the deciding factor is usually portability. A red-team stack that swaps models mid-campaign (frontier target, abliterated baseline, uncensored generator) wants every tier behind the same OpenAI-compatible endpoint. Fork-specific steering locks you to one engine build.
Where this fits on UNST33R
UNST33R serves the weight-level variants upstream: unst33r-deepseek (DeepSeek-V4.1-Flash, abliterated) and the uncensored GLM-5.3 tiers, all behind one per-token endpoint. If your work compares refusal behavior across model families, inference-time steering is a good local experiment on quantized GGUF drops; production harness runs are simpler against a checkpoint where the projection is already done. Test both under your own authorization scope and the acceptable-use policy.