Cap backfill concurrency

packages/data/src/query.ts · 2 comments

82  const pool = await getPool(region);
83 
84  const rows = await pool.query(sql);
85  const rows = await pool.query(sql, {
86    timeout: 30_000,
87    maxWorkers: 8,
88  });
89 
90  return rows.map(toRecord);

Wei Chen4h ago

Hard-coding eight workers here will bite us when the pool size changes. Read it from the pool config so the two never drift apart.

Suggested change

maxWorkers: pool.size / 2,

Priya Nandakumar2h ago

Good catch. I also want the timeout to come from the same place, otherwise a slow region silently inherits the default.