DEV Community

Cover image for Understanding Wallet Sync in the Midnight SDK: Why Your Deploy Fails Before It Starts
Harrie
Harrie

Posted on

Understanding Wallet Sync in the Midnight SDK: Why Your Deploy Fails Before It Starts

#blockchain #midnightfordevs #tutorial

You call balanceUnboundTransaction. It throws — or worse, it succeeds silently with wrong token balances, and the transaction fails somewhere downstream with a message that has nothing to do with sync. You check the obvious things. Funds are in the wallet. The contract compiled. The proof server is running. Nothing looks wrong.

The problem is that your wallet hasn't finished scanning the chain, and balanceUnboundTransaction built a transaction from an incomplete local database. This is probably the most common early failure in Midnight development. It's almost always invisible until you've seen it once.

The mental model shift

Most developers approach a crypto wallet like a stateless lookup tool — you call it, it queries the network, you get a live answer. Midnight's wallet doesn't work that way.

The Midnight wallet is a local database. Before it can do anything useful, it has to scan the blockchain from its last known position (or from genesis for a fresh wallet) to discover which UTXOs and notes belong to it. Until that scan completes, the wallet has an incomplete, potentially stale view of its own balances.

Shielded transactions make this more expensive than other chains. Every shielded note on Midnight is encrypted. The wallet has no shortcut — it has to attempt to decrypt every note it encounters to discover which ones it owns. On a busy testnet, that's a lot of blocks.

balanceUnboundTransaction pulls from this local database. Pass it before the scan completes and it builds a transaction from whatever partial state it has. On a fresh wallet with nothing scanned, it finds no UTXOs and errors. After partial sync, it might find stale UTXOs, build a valid-looking transaction, and have that transaction rejected at submission time. Neither failure is easy to debug from the error message alone.

The three sub-wallets

WalletFacade wraps three independent sub-wallets, each scanning for different things and completing at different speeds.

Shielded wallet manages private NIGHT tokens via Zswap notes. Every block it encounters requires trial decryption against your secret keys. This is the slowest of the three — on a cold testnet start with hundreds of blocks to scan, expect 30–60 seconds. The state path is state.shielded.state.progress.

Unshielded wallet handles transparent UTXOs, visible on-chain without ZK protection. Scanning is faster because it can filter by address without decryption. The state path is state.unshielded.progress — note the different nesting from shielded.

DUST wallet manages tDUST fee tokens via UTXO aggregation. On an active chain it's near-instant. On an idle chain (a local devnet with no recent transactions), it hits a documented bug. The state path is state.dust.state.progress — back to the extra .state level, like shielded.

All three need to complete before balanceUnboundTransaction can do its job. balanceUnboundTransaction needs shielded keys to find your private notes, and the DUST wallet to cover fees. Missing either one and it either errors or produces bad output.

The state shape: what facade.state() actually emits

This is the part of the API that trips people up most — the three sub-wallets have inconsistent state shapes:

// These three paths are NOT symmetric
state.shielded.state.progress    // shielded: extra .state level
state.unshielded.progress        // unshielded: flatter
state.dust.state.progress        // dust: extra .state level, like shielded
Enter fullscreen mode Exit fullscreen mode

If you try to check sync completion by calling .isStrictlyComplete() directly on the wrong path, you get undefined, which is falsy, which means your sync check always fails — but silently.

The official Midnight CLI code uses a defensive wrapper for exactly this reason:

const isProgressStrictlyComplete = (progress: unknown): boolean => {
  if (!progress || typeof progress !== 'object') return false;
  const candidate = progress as { isStrictlyComplete?: unknown };
  if (typeof candidate.isStrictlyComplete !== 'function') return false;
  return (candidate.isStrictlyComplete as () => boolean)();
};
Enter fullscreen mode Exit fullscreen mode

This handles SDK versions where isStrictlyComplete might not exist on the progress object, and accounts for return values that are boolean | undefined rather than a clean boolean. Call it with each sub-wallet's progress field and you get a safe, consistent answer.

isSynced vs isStrictlyComplete() — they're not the same thing

WalletFacade exposes a top-level isSynced flag. It looks like the right thing to check, and it's tempting to use it. The problem: it's a convenience property that can return true while the DUST wallet's isStrictlyComplete() is still false.

If you gate your transaction on isSynced alone, you might proceed with a wallet that can't cover fees. The transaction builds, passes balanceUnboundTransaction, and fails at submission with an opaque message about DUST.

Midnight's own official sync code doesn't use isSynced. It checks each sub-wallet individually:

Rx.filter(
  (state: FacadeState) =>
    isProgressStrictlyComplete(state.shielded.state.progress) &&
    isProgressStrictlyComplete(state.dust.state.progress) &&
    isProgressStrictlyComplete(state.unshielded.progress),
),
Enter fullscreen mode Exit fullscreen mode

If you want production reliability, follow the official code, not the convenience flag.

The correct sync pattern

Here's syncWallet from Midnight's Bulletin Board CLI tutorial — the canonical implementation straight from the docs:

export const syncWallet = (
  logger: Logger,
  wallet: WalletFacade,
  throttleTime = 2_000,
  timeout = 90_000,
) => {
  logger.info('Syncing wallet...');
  return Rx.firstValueFrom(
    wallet.state().pipe(
      Rx.tap((state: FacadeState) => {
        const shieldedSynced = isProgressStrictlyComplete(state.shielded.state.progress);
        const unshieldedSynced = isProgressStrictlyComplete(state.unshielded.progress);
        const dustSynced = isProgressStrictlyComplete(state.dust.state.progress);
        logger.debug(
          `Sync progress: shielded=${shieldedSynced}, unshielded=${unshieldedSynced}, dust=${dustSynced}`,
        );
      }),
      Rx.throttleTime(throttleTime),
      Rx.filter(
        (state: FacadeState) =>
          isProgressStrictlyComplete(state.shielded.state.progress) &&
          isProgressStrictlyComplete(state.dust.state.progress) &&
          isProgressStrictlyComplete(state.unshielded.progress),
      ),
      Rx.tap(() => logger.info('Sync complete')),
      Rx.timeout({
        each: timeout,
        with: () => Rx.throwError(() => new Error(`Wallet sync timeout after ${timeout}ms`)),
      }),
    ),
  );
};
Enter fullscreen mode Exit fullscreen mode

Rx.throttleTime(2_000) is there because wallet.state() emits on every block — potentially multiple times per second during active sync. Without throttling, the filter runs on every emission and any side effects (logging, UI updates) fire hundreds of times per sync. The tap before throttleTime still logs every emission at DEBUG level, but the filter only evaluates every 2 seconds.

Rx.timeout({ each: timeout, ... }) — the each key applies per-emission gap, not total elapsed time. If the wallet emits on schedule (every 2 seconds from throttling), each: 90_000 only fires if 90 seconds pass with no emission at all — meaning the wallet has gone completely silent. That's the right semantics: catch stuck wallets, not just slow ones.

Rx.firstValueFrom converts the observable into a Promise that resolves as soon as the filter passes. Easy to await in async code without managing subscriptions manually.

The DUST wallet bug on idle chains

On chains with little or no recent transaction activity — a local devnet you just started, or an integration test environment — the DUST wallet's isStrictlyComplete() never returns true. The progress tracker doesn't see any DUST consolidation events, so it never records completion. syncWallet hangs until the timeout fires.

The workaround is to skip the DUST strict-completion requirement in development environments:

const syncWalletWithFallback = async (
  wallet: WalletFacade,
  isDev = false,
  timeoutMs = 90_000,
): Promise<void> => {
  await Rx.firstValueFrom(
    wallet.state().pipe(
      Rx.throttleTime(2_000),
      Rx.filter((state: FacadeState) => {
        const shieldedReady = isProgressStrictlyComplete(state.shielded.state.progress);
        const unshieldedReady = isProgressStrictlyComplete(state.unshielded.progress);
        const dustReady = isProgressStrictlyComplete(state.dust.state.progress);
        // On idle devnets, DUST never completes — skip that check in dev
        return shieldedReady && unshieldedReady && (dustReady || isDev);
      }),
      Rx.timeout({
        each: timeoutMs,
        with: () => Rx.throwError(() => new Error(`Sync timeout after ${timeoutMs}ms`)),
      }),
    ),
  );
};
Enter fullscreen mode Exit fullscreen mode

Don't use this shortcut in production. If DUST sync is incomplete on mainnet or testnet, fee payments will fail.

balanceUnboundTransaction — the full picture

Every competitor article treats balanceUnboundTransaction as a one-line call. The actual signature is:

const recipe = await wallet.balanceUnboundTransaction(
  tx,                                              // the unbound transaction from your circuit
  {
    shieldedSecretKeys: this.zswapSecretKeys,      // Zswap keys (plural — an array)
    dustSecretKey: this.dustSecretKey,             // DUST key (singular)
  },
  { ttl },                                         // time-to-live for the transaction
);
Enter fullscreen mode Exit fullscreen mode

Two separate key types: shieldedSecretKeys is an array of Zswap keys that unlock your shielded notes, and dustSecretKey is a single key for the DUST fee wallet. If you pass the wrong key type or omit one, the call fails — or worse, builds a transaction that covers the wrong balance. This is separate from sync, but it's the other common reason this call fails.

balanceUnboundTransaction also doesn't submit the transaction. It returns a recipe. You need two more steps:

// Step 1: Balance the unbound transaction
const recipe = await wallet.balanceUnboundTransaction(tx, keys, { ttl });

// Step 2: Finalize the recipe into a submittable transaction
const finalized = await wallet.finalizeRecipe(recipe);

// Step 3: Submit
const txHash = await wallet.submitTransaction(finalized);
Enter fullscreen mode Exit fullscreen mode

This three-step flow matters because finalizeRecipe is where ZK proofs get generated. If you're debugging a failure and it happens in finalizeRecipe rather than balanceUnboundTransaction, the problem is in proof generation, not sync.

The full startup sequence

From Midnight's own CLI tutorial, the full sequence looks like this:

// 1. Create the wallet provider and get the facade
const walletProvider = await MidnightWalletProvider.build(logger, envConfig, seed);
const wallet: WalletFacade = walletProvider.wallet;

// 2. Start the wallet (begins network connections and scanning)
await walletProvider.start();

// 3. If this is a first-time setup: wait for unshielded funds and generate DUST
//    Skip this if the wallet already has DUST from a previous run.
const unshieldedState = await waitForUnshieldedFunds(logger, wallet, envConfig);
await generateDust(logger, seed, unshieldedState, wallet);

// 4. Wait for full sync before any transactions
await syncWallet(logger, wallet);

// 5. Now safe to call balanceUnboundTransaction
Enter fullscreen mode Exit fullscreen mode

Steps 3 is first-run setup: generateDust creates the initial tDUST that fees will be paid from. On subsequent runs with a funded wallet, skip it. The key point is that step 4 — syncWallet — must always happen before step 5. The start() call in step 2 begins scanning, but does not wait for completion. There's always a gap.

Three failure modes

When balanceUnboundTransaction runs before sync completes, the failure depends on how far along sync is:

Not started: The wallet has no UTXOs indexed at all. You'll get an immediate error about insufficient balance or missing UTXOs. Annoying but at least clear.

Partial sync: The wallet has some UTXOs but not all. balanceUnboundTransaction picks the UTXOs it knows about, which may be stale or already spent. The call succeeds, but the transaction gets rejected at submission time with a message about invalid inputs or double-spend.

Shielded-only sync: Shielded wallet is complete, unshielded and DUST are not. Transaction builds with correct private balance but fails on fee payment with "insufficient DUST balance" — even if the faucet topped you up.

The second failure mode is the worst to diagnose. You have a finalized transaction, you submitted it, and it fails with a message that doesn't mention sync at all.

Production patterns

Always gate transactions on sync. Even after the initial startup sync, wallets can fall behind if the node connection drops or the chain catches up. Check before every balanceUnboundTransaction, not just at startup:

async function safeBalance(
  wallet: WalletFacade,
  tx: UnboundTransaction,
  keys: WalletKeys,
): Promise<Recipe> {
  // Confirm sync first, re-sync if needed
  const state = await Rx.firstValueFrom(wallet.state().pipe(Rx.take(1)));
  if (!state.isSynced) {
    await syncWallet(logger, wallet);
  }
  return wallet.balanceUnboundTransaction(tx, keys, { ttl: ttlOneHour() });
}
Enter fullscreen mode Exit fullscreen mode

Re-sync on transaction failure. If submission fails with UTXO-related errors, the local database may have fallen behind. Re-sync before retrying:

async function submitWithRetry(
  wallet: WalletFacade,
  tx: UnboundTransaction,
  keys: WalletKeys,
  maxRetries = 3,
): Promise<string> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      if (attempt > 0) {
        await syncWallet(logger, wallet, 2_000, 30_000);
      }
      const recipe = await wallet.balanceUnboundTransaction(tx, keys, { ttl: ttlOneHour() });
      const finalized = await wallet.finalizeRecipe(recipe);
      return wallet.submitTransaction(finalized);
    } catch (err) {
      if (attempt === maxRetries) throw err;
      logger.warn(`Attempt ${attempt + 1} failed: ${(err as Error).message}. Re-syncing...`);
    }
  }
  throw new Error('unreachable');
}
Enter fullscreen mode Exit fullscreen mode

Monitor sync state continuously. In a long-running service, subscribe to wallet state and alert when the wallet desyncs:

wallet.state().subscribe({
  next: (state) => {
    if (!state.isSynced) {
      logger.warn('Wallet lost sync — queuing transactions until recovery');
      // block new transaction requests, drain the in-flight queue
    }
  },
  error: (err) => logger.error('Wallet state subscription error:', err),
});
Enter fullscreen mode Exit fullscreen mode

Common pitfalls: wrong and right

Wrong — calling balanceUnboundTransaction immediately after start():

await walletProvider.start();
// ❌ start() doesn't wait for sync
const recipe = await wallet.balanceUnboundTransaction(tx, keys, { ttl });
Enter fullscreen mode Exit fullscreen mode

Right — wait for sync before any transaction work:

await walletProvider.start();
await syncWallet(logger, wallet); // ✅ blocks until all three sub-wallets are complete
const recipe = await wallet.balanceUnboundTransaction(tx, keys, { ttl });
Enter fullscreen mode Exit fullscreen mode

Wrong — checking isSynced and trusting it for DUST:

const state = await Rx.firstValueFrom(wallet.state().pipe(Rx.take(1)));
if (state.isSynced) {
  // ❌ isSynced can be true while DUST's isStrictlyComplete() is false
  await wallet.balanceUnboundTransaction(tx, keys, { ttl });
}
Enter fullscreen mode Exit fullscreen mode

Right — check each sub-wallet individually:

const state = await Rx.firstValueFrom(wallet.state().pipe(Rx.take(1)));
const allSynced =
  isProgressStrictlyComplete(state.shielded.state.progress) &&
  isProgressStrictlyComplete(state.unshielded.progress) &&
  isProgressStrictlyComplete(state.dust.state.progress);
if (allSynced) {
  await wallet.balanceUnboundTransaction(tx, keys, { ttl });
}
Enter fullscreen mode Exit fullscreen mode

Wrong — calling .isStrictlyComplete() directly without the defensive wrapper:

// ❌ isStrictlyComplete can be undefined in some SDK versions
const dustReady = state.dust.state.progress.isStrictlyComplete();
Enter fullscreen mode Exit fullscreen mode

Right — use the defensive helper:

// ✅ handles undefined, null, and function-not-present cases
const dustReady = isProgressStrictlyComplete(state.dust.state.progress);
Enter fullscreen mode Exit fullscreen mode

Wrong — treating balanceUnboundTransaction as a submit call:

// ❌ this only balances the transaction; it doesn't submit it
await wallet.balanceUnboundTransaction(tx, keys, { ttl });
// nothing submitted, no error, silently did nothing useful
Enter fullscreen mode Exit fullscreen mode

Right — use the full three-step flow:

const recipe = await wallet.balanceUnboundTransaction(tx, keys, { ttl }); // ✅ balance
const finalized = await wallet.finalizeRecipe(recipe);                     // ✅ prove
const txHash = await wallet.submitTransaction(finalized);                  // ✅ submit
Enter fullscreen mode Exit fullscreen mode

Quick reference

Midnight's wallet is a local database rebuilt by scanning the chain. balanceUnboundTransaction reads that database, so calling it before sync means building transactions from incomplete data.

The three sub-wallets sync independently and have inconsistent state shapes — check each one explicitly with the isProgressStrictlyComplete helper rather than trusting the top-level isSynced flag.

The sequence, from Midnight's own CLI code:

  1. await walletProvider.start() — begins scanning, does not wait for it
  2. await syncWallet(logger, wallet) — blocks until all three sub-wallets are strictly complete
  3. balanceUnboundTransaction → finalizeRecipe → submitTransaction — three separate steps

On idle devnets, DUST sync will never complete — use the fallback that skips it in dev. In production, always require all three.

TypeScript patterns in this article are derived from Midnight's official Bulletin Board CLI tutorial. Wallet SDK API based on `@midnight-ntwrk/midnight-js-` packages.*

Top comments (0)