Skip to content
JournalTORIS / FIELD NOTES

← Blog Project notes

A Dead-Link Checking Batch: Not Treating a Failed Response as Grounds for Deletion

Looking back at content management and dead-link check automation: telling a temporary failure of an external link from a real shutdown, and leaving evidence for a human to judge.

2 min read Read the Korean original

Joohwan Yu · Founder, Full Stack Developer

Toris · Published

Contents

In content management work, I handled a batch that checks for dead links. The goal was to cut down the repetitive work of a person opening links one by one to see if they were alive. At first it looks like sending HTTP requests and collecting the results would be enough, but interpreting failures from external systems was harder than that.

A failed response is different from content being gone

A timeout may just mean the other server is slow. Restricted access may mean it blocks automated requests. And the mere presence of a redirect doesn’t mean the original content can’t be viewed. If you reduce the check result to a simple true or false, the reason for the operator to check again disappears.

Looking back on that experience, I arrived at the standard that automation is good at observation and the final judgment is separate. Linking every failure straight to deletion or unpublishing goes too far. For links outside the service in particular, I can’t control the server’s state, so the uncertainty of the result has to be preserved.

How I would split it now

Observed resultNext action the operator can take
Normal responseUpdate the last-checked time
Timeout or temporary errorRecheck after an interval
Access restrictedSeparate out as a target for manual browser check
Repeated not-found responseReview whether the original has ended

This table is not a reproduction of the exact implementation of the batch at the time. It is the classification standard I would apply if I built the same feature again. A person can continue the judgment only if there is observed evidence such as the response code, the check time, and the final destination address.

The results of automation also need review

Checking many addresses at once can put a burden on both the other site and our own system. Now I would state the concurrency and the time limit explicitly, and decide where to resume if it stops midway. I would also report separately that the batch itself succeeded and that every link was checked.

The value of this experience for me is not in a number like how many hours automation saved. I can’t attach such a figure if I didn’t measure it separately. Instead, I learned the direction of turning work that people repeatedly opened by hand into a checkable list. Automation was more trustworthy as a tool that steadily gathers the observations needed for judgment than as a tool that rushes to conclusions.

Related post: Scope of change on an operating project

Advertisement