> Source: https://glassmkr.com/for-storage # Your drive says it is healthy. That is not the same as being healthy. Every drive carries a one-bit self-assessment, and it is the field most monitoring checks. We measured how far that bit can sit from reality: 477 dead sectors, and it still read PASSED. [Install in 2 minutes](https://app.glassmkr.com/register) [View Crucible on GitHubMIT](https://github.com/glassmkr/crucible) 3 nodes free. Point it at one storage box before deploying fleet-wide. `curl -fsSL https://glassmkr.com/install.sh | sudo bash` Crucible v0.15.1 on npm ## The problem The SMART overall-health verdict is a pass or fail against normalised attribute values, not against the raw counts an operator reads. The manufacturer sets those thresholds where it expects warranty-relevant death. A drive can work through most of its spare sector pool and still sit comfortably inside them. We ran a drive-health campaign across thirteen boxes to find out how wide that gap gets. One 4 TB drive was carrying **477 reallocated sectors** and reported its own health as **PASSED in all 140 readings** we logged. A full destructive write-and-verify pass over that same drive came back clean, zero errors, because reallocation exists precisely to hide the damage from the host. The only place it was visible was the raw attribute counter, and only to something watching that counter over time. That is the whole job. Not reading the verdict; reading what is underneath it, on a schedule, and knowing which of those numbers mean something. [Read the full write-up, with the data](/blog/smart-said-passed). ## How Glassmkr fits ### Sixteen rules for the storage layer alone. Seven for drives and controllers, three for ZFS, six for filesystem and capacity. They run on every collection cycle against every drive the box can see, and they carry the remediation with them rather than leaving you to work out what a raw attribute number means. ### We alert on the counters that latch, and not on the ones that flap. Reallocated sectors only ever go up, so they are safe to page on. Pending sectors oscillate by design: on that same drive ours swung between 0 and 96 while the actual damage never moved. An alert keyed on pending would have fired and self-cleared three times in under a day. We considered it and the data talked us out of it. ### Drives behind a hardware RAID controller are still drives. A controller that presents one logical volume hides the physical disks from most tooling. Crucible enumerates them through the controller and reads each one's SMART directly, so a failing member is visible before the array degrades rather than after. ### ZFS gets its own rules, not a generic disk check. Pool health, scrub errors and a faulted SLOG are three different problems with three different responses. Treating them as one "disk problem" is how a degraded pool sits unnoticed while somebody investigates capacity. ### Capacity is a trend, not a threshold. A disk at 71 percent that has been at 71 percent for a year is fine. A disk at 71 percent climbing steadily is a scheduled outage. We project the fill rather than waiting for a percentage line to be crossed at 3am. ### An unreadable drive is reported, not skipped. If SMART cannot be read for a disk, whether the controller does not support it or the tooling is missing, that is stated as its own condition. A drive we cannot see is not a drive that is fine, and it should never silently disappear from a health summary. ## What we do not claim We found one failing drive in that campaign and we found it already at 477, so we never watched it climb from zero. That means we cannot tell you what count is dangerous. Is a drive at 3 reallocated sectors in trouble, or fine for another four years? Our data cannot answer that, and any threshold we published from a single drive would be an invention with a number attached. So reallocated sectors stay critical rather than being tiered by size, because the two ways of being wrong are not equally expensive: telling you to investigate a drive that is actually dying costs you data, and paging you about one that turns out to be stable costs you a ticket. ## The agent you can read Crucible is MIT licensed and the repository is public. It runs as an unprivileged user, and the few commands that need root go through a narrow root-owned wrapper with a fixed argument list per action: no shell, no caller-supplied arguments. You can read exactly what it collects and exactly what leaves the box. [What we collect and what we do not](/trust) is documented separately, in detail. ## Pricing reminder 3 nodes free, forever. $3 per node per month beyond that. No per-drive or per-metric charge. A 60-bay JBOD costs the same as a single-disk VPS. ## Point it at one storage box first. Install takes two minutes and the free tier covers three nodes, so you can see what it says about hardware you already know the history of before you trust it with the rest. [Get started free](https://app.glassmkr.com/register) [Read the docs](/docs)