Build the lookup once, before the loop
How repeated full-list scans turn a small catalogue into expensive work.
When the project growsAbout 2 min read
The helper hides the second scan
A catalogue parser processes each row. Inside that loop, a helper scans all rows again to count how often an employer URL is used. The helper looks small, but it repeats the whole-list work for every row.
For an illustrative list of N rows, N full scans give roughly N × N comparisons: O(N²). Ten rows suggest about 100 comparisons; 500 suggest about 250,000. These are explanatory counts, not measured instructions, timings or a claim that 500 rows always exceed a server limit.
One shared index
My Next Step’s catalogue code builds shared URL usage once and reuses it while parsing rows. A Map can hold a normalised URL and its count. With typical efficient map lookups, counting and processing each take a pass, rather than repeating a full scan for every row.
The key must mean the same thing in both passes. If one path normalises trailing slashes and another does not, the faster code can classify differently. The change also trades some memory for less repeated computation; it does not make every operation free.
// One pass to count, then reuse the result.
const uses = new Map();
for (const row of rows) {
uses.set(row.url, (uses.get(row.url) ?? 0) + 1);
}
for (const row of rows) {
const shared = uses.get(row.url) > 1;
// Process this row using the same normalised URL key.
}Measure the growth
Test representative sizes and compare output as well as work performed. A useful regression check can count index construction or scan calls instead of relying only on a fragile wall-clock threshold. Measure actual runtime separately.
A nested loop is not automatically wrong. The question is whether it repeats avoidable work on a growing collection. Look inside helpers and callbacks as well as the visible loop before asking for a larger server.