YM Guide's Operational Debt Was in Data Refresh, Not Recommendations
The limits of manual updates while building youth policy curation. Separating recommendation accuracy from information freshness, and empty results from collection failures.
Joohwan Yu · Founder, Full Stack Developer
Toris · Published
Contents
YM Guide was a product meant to narrow down youth policy information through questions and recommendations. Considering the call limits of the source API, I chose a structure that stores and serves processed data. It had the benefit of not querying the source on every request, but in return I was responsible for refreshing the data.
Fast Responses and Fresh Information Were Different Axes
Querying stored data simplifies the response path. But if the stored policies are stale, the result can quickly help people make wrong decisions. A policy’s application period and conditions are information that must be valid before any recommendation score.
In the earlier retrospective I stated plainly that refresh being manual was operational debt. Even now, I don’t want to rephrase that part as if it were completed automation. Manual handling can be a choice that speeds up launch, but if who checks and when is missing, it becomes forgotten work rather than repeated work.
Don’t Hide Collection Failures Behind an Empty List
The public example at the time had code that returned an empty array when the API failed. It has the advantage of not stopping the whole screen, but looking at it now, it is hard to tell “no matching policy” from “couldn’t fetch policies.” Users need different guidance in those two situations.
If I rebuilt it, I would separate the last successful data from the collection status. Even if existing information can be shown when a refresh fails, the time of the last confirmation must be recorded. Whether to keep showing stale data or to restrict some recommendations should be decided according to the nature of the information. Showing the cache unconditionally is not the answer.
If I Automate, Start With the Success Criteria
A scheduler having run does not mean the refresh is complete. You also have to check whether the data dropped sharply from what was expected, whether required fields disappeared, and whether expired policies remain. If I did it now, I would distinguish collection, validation, and publishing, and I would also log in the operations record the cases where existing data was kept because validation failed.
Looking back at this product, I learned that you must not focus only on the recommendation algorithm. Sophisticated sorting on top of stale input does not build trust. If I were choosing the priorities for the next piece of work, before tuning one more weight I would first check when and how the data gets refreshed.
Related record: YM Guide manual refresh and recommendation design
Advertisement