DEV Community

LewiX-369
LewiX-369

Posted on

Bringing External Data On-Chain: Oracle Patterns for Midnight

Spent last weekend on Midnight Network bounties because the payouts are real and the tutorial backlog is long. Bounty #304 is about oracles. Or more accurately, about the fact that Midnight doesn't have one in the Chainlink sense, and someone needs to write up the patterns devs actually use instead. So that's what this is.

If you've been gluing Pyth or Chainlink price feeds into Solidity for a few years, the way Midnight handles external data is going to feel sideways at first. There's no oracle smart contract you can call. No round IDs. No aggregation network. Instead you have a witness system and a TypeScript glue layer, and the trust model lives partly off-chain whether you like it or not. Three patterns came out of a couple days of poking at this. They cover the territory from "scrappy admin push" to "compose two smart contracts that don't trust each other".

All three smart contracts in here compile clean against the public playground compiler at compact-playground.up.railway.app, language version 0.22, compiler 0.30.0. I'll get to why I ended up running them through the playground API instead of the Docker image further down. Short version: Docker wouldn't pull and I wasn't burning another hour on it.

Before you start

You'll need Node 20+, TypeScript 5+, and the Midnight SDK packages (@midnight-ntwrk/compact-js, @midnight-ntwrk/indexer-client). The Compact code runs on language version 0.22 with compiler 0.30.0 — use the public playground at compact-playground.up.railway.app to test without a local toolchain.

If you're new to Midnight, read the getting started guide first. The patterns here assume you can deploy a smart contract and call a circuit. The getting started guide covers both before this becomes useful.

Why Midnight has no native oracle

Zero-knowledge circuits are deterministic. Same inputs, same output, every time. That property is not optional, it's the whole reason a verifier can trust a proof without re-running the computation. If you stuffed an HTTP fetch inside a circuit, you'd get a different answer every time you generated a proof, and the proof would stop being a proof.

So Midnight pushes the data fetch out of the circuit and into the prover's TypeScript layer. The mechanism for this is called a witness. A witness is a function the circuit declares but doesn't define. The TypeScript code that wraps the smart contract supplies the implementation at proof-generation time. The circuit then operates on whatever values come back as if they were private inputs.

This is fine. It's also where every oracle question on Midnight has to start. The circuit cannot verify that the witness told the truth. It can only verify that whatever the witness said, the circuit applied its rules correctly. So the trust question is: who do you trust to fill in the witness, and how do you constrain what they can say?

That's the framing for all three patterns below.

A note before code. Compact treats every parameter passed into a circuit as a private witness value by default. If you try to write one of those values into a public ledger field without explicitly calling disclose() on it first, the compiler refuses. Hard error, not a warning. The exact text is something like:

potential witness-value disclosure must be declared but is not:
witness value potentially disclosed: the value of parameter X of exported circuit Y
nature of the disclosure: ledger operation might disclose the witness value via this path through the program
Enter fullscreen mode Exit fullscreen mode

I burned ten minutes on this the first time. The fix is just to wrap the assignment, field = disclose(param). But if you've come from Solidity where the runtime barrier between "this argument is public input" and "this argument is private" doesn't exist, the error is genuinely confusing. It's also Compact doing the right thing. ZK circuits leak information through state writes; the compiler is forcing you to declare your intent to leak so a reviewer can audit it.

OK. Code.

Pattern 1. The admin push.

Simplest possible oracle. Pick a wallet, give it write access to a ledger field, ship it.

// pattern1-admin-push.compact
pragma language_version >= 0.22;
import CompactStandardLibrary;

export ledger admin: Bytes<32>;
export ledger price: Uint<64>;
export ledger updatedAtBlock: Uint<32>;

export circuit initialize(adminKey: Bytes<32>): [] {
    admin = disclose(adminKey);
    return [];
}

export circuit push_price(newPrice: Uint<64>, blockHeight: Uint<32>): [] {
    const caller = ownPublicKey().bytes;
    assert(caller == admin, "Admin only");
    price          = disclose(newPrice);
    updatedAtBlock = disclose(blockHeight);
    return [];
}

export circuit get_price(): Uint<64> {
    return price;
}

export circuit get_price_with_block(): Uint<32> {
    return updatedAtBlock;
}
Enter fullscreen mode Exit fullscreen mode

The exported admin, price, and updatedAtBlock are public ledger fields. Anyone who reads the smart contract's state, including consumers, sees them. The initialize circuit gets called once at deploy time to register the admin's key. After that, only the admin can push.

Access control is one line: const caller = ownPublicKey().bytes; assert(caller == admin, "Admin only");. The ownPublicKey() call returns a ZswapCoinPublicKey, and .bytes peels the raw 32-byte representation off it. Compare against the stored admin bytes, refuse if it doesn't match.

There's a footgun here that the Midnight team has actually flagged as bounty #295. ownPublicKey() returns whoever is generating the ZK proof for this circuit call. That's not necessarily who paid for the transaction or who signed the Zswap layer. In a single-wallet admin scenario you control, this distinction doesn't matter. In a more complex scenario where you're conflating "who proves" with "who pays", you can get bitten. For Pattern 1's threat model the simple check is fine, but if you're using this pattern for anything more sensitive than a dashboard, go read #295 first.

The TypeScript side is what you'd guess.

import { Contract } from '@midnight-ntwrk/compact-js';
import { priceFeedAbi } from './contract/index.js';

async function pushPrice(
  contract: Contract<typeof priceFeedAbi>,
  priceInCents: bigint,
) {
  const blockHeight = BigInt(await provider.getBlockHeight());

  const tx = await contract.callCircuit('push_price', {
    newPrice:    priceInCents,
    blockHeight: blockHeight,
  });

  const submitted = await tx.submit();
  console.log('Price updated at tx:', submitted.hash);
}
Enter fullscreen mode Exit fullscreen mode

And the consumer side, where the freshness check lives:

async function readPrice(
  contract: Contract<typeof priceFeedAbi>,
) {
  const state = await contract.queryState();
  const { price, updatedAtBlock } = state;

  const currentBlock = BigInt(await provider.getBlockHeight());
  const staleness = currentBlock - updatedAtBlock;

  if (staleness > 100n) {
    throw new Error(`Price is ${staleness} blocks old, too stale`);
  }

  return price;
}
Enter fullscreen mode Exit fullscreen mode

Note the staleness check happens client-side. The smart contract itself does not enforce a freshness window. You can move that into the circuit by adding a currentBlock parameter and asserting currentBlock - updatedAtBlock < N, but most consumers just gate on staleness in their own logic and skip the on-chain check.

What this gets you. One key, one writer, public reads, optional staleness gate. Cheap to deploy, fast to ship, fine for an internal dashboard or a treasury display.

What this does not get you. Anything resembling actual oracle security. If the admin key leaks, every consumer is reading attacker-controlled prices. There's no proof on chain that the value matches any external market. The chain has no way to detect the admin pushing a wrong number. Anti-replay is not a thing because there's no signature to replay.

Trust model in one line: you trust whoever holds the admin key. Period. Use it for prototypes. Don't use it for anything that holds value.

Pattern 2. Sign it.

Pattern 1 collapses if the admin key is compromised. Pattern 2 separates the data provider's identity from on-chain access control. The provider runs an oracle service, signs each (price, sequence) pair with a private key, and publishes the signed payload at a known endpoint. The TypeScript witness layer fetches the latest payload, verifies the signature against the registered oracle key, and feeds the verified values into the circuit. The circuit itself doesn't need an admin check anymore. Anyone can call the update circuit. The proof only validates if the witness layer produced a fresh, signed pair.

// pattern2-witness-signed.compact
pragma language_version >= 0.22;
import CompactStandardLibrary;

export ledger oracleKey: Bytes<32>;
export ledger price: Uint<64>;
export ledger lastSeq: Uint<64>;

struct PriceUpdate {
    price: Uint<64>;
    seq:   Uint<64>;
}

// TypeScript fetches the signed payload, verifies the signature off-chain,
// then surfaces the (price, seq) pair here.
witness signed_price_data(): PriceUpdate;

export circuit initialize(key: Bytes<32>): [] {
    oracleKey = disclose(key);
    return [];
}

export circuit update_verified_price(): [] {
    const update = signed_price_data();
    assert(disclose(update.seq) > lastSeq, "Sequence number must increase");
    price   = disclose(update.price);
    lastSeq = disclose(update.seq);
    return [];
}

export circuit get_price(): Uint<64> {
    return price;
}
Enter fullscreen mode Exit fullscreen mode

The struct PriceUpdate block matters. My first attempt used a tuple type because that's the obvious shape, Tuple<Uint<64>, Uint<64>>. Compiler ate it on the witness declaration but rejected it on the destructure. The error was:

parse error: found "(" looking for a const binding
Enter fullscreen mode Exit fullscreen mode

at the const (price, seq) = signed_price_data(); line. Then I tried update.0 style indexing, which Solidity tuples kind of support if you bend them. Compact will not have it:

parse error: found "0" looking for an identifier
Enter fullscreen mode Exit fullscreen mode

Struct types with named fields work cleanly. So that's what's there. Probably the right call anyway because named fields read better than positional indexing six months from now.

The on-chain assertion is assert(disclose(update.seq) > lastSeq, "Sequence number must increase"). This is the anti-replay guard. The oracle service maintains a strictly increasing counter. Every signed payload includes the current value of that counter. If an attacker captures a legitimately signed payload and tries to feed it back into the witness later, the smart contract sees that the sequence has already been consumed and rejects the proof. You cannot replay yesterday's signed price into today's smart contract state.

Note what's NOT happening on-chain. The signature verification. As of language version 0.22 the Compact standard library does not include a signature verification primitive for secp256k1, EdDSA, or anything else useful. There's no ecVerify(pubkey, message, signature) you can call from a circuit. So the verification has to live in the TypeScript witness layer.

import { ethers } from 'ethers';

const ORACLE_API = 'https://your-oracle-service.example.com/latest';
const ORACLE_PUBLIC_ADDRESS = '0xYourOracleAddress...';

async function signed_price_data(): Promise<{ price: bigint; seq: bigint }> {
  const response = await fetch(ORACLE_API);
  const { price, seq, signature } = await response.json();

  const message = ethers.utils.solidityKeccak256(
    ['uint64', 'uint64'],
    [BigInt(price), BigInt(seq)]
  );

  const recovered = ethers.utils.recoverAddress(message, signature);
  if (recovered.toLowerCase() !== ORACLE_PUBLIC_ADDRESS.toLowerCase()) {
    throw new Error('Oracle signature verification failed');
  }

  return { price: BigInt(price), seq: BigInt(seq) };
}

const tx = await contract.callCircuit(
  'update_verified_price',
  {},
  { witnesses: { signed_price_data } }
);
await tx.submit();
Enter fullscreen mode Exit fullscreen mode

The oracle service itself is not particularly clever:

import express from 'express';
import { ethers } from 'ethers';

const oracleWallet = new ethers.Wallet(process.env.ORACLE_PRIVATE_KEY!);
let seqCounter = 0n;

app.get('/latest', async (req, res) => {
  const price = await fetchPriceFromExternalSource();
  const seq   = ++seqCounter;

  const message = ethers.utils.solidityKeccak256(
    ['uint64', 'uint64'],
    [price, seq]
  );
  const signature = await oracleWallet.signMessage(
    ethers.utils.arrayify(message)
  );

  res.json({ price: price.toString(), seq: seq.toString(), signature });
});
Enter fullscreen mode Exit fullscreen mode

The seqCounter lives in process memory in this snippet, which is fine for a demo, terrible for production. In a real deployment you'd persist it. If the service restarts and the counter resets to zero, your next signed update gets rejected by every consumer because sequence didn't advance. Same outcome if you accidentally run two instances of the service against the same key, they'll race on the counter and produce conflicting payloads. Use a single writer with persistent state.

Net of all this. The oracle's signing key and the deployer wallet are now decoupled, anyone can trigger updates, replay is genuinely dead because of the sequence assertion. The signing key can sit in a KMS and stay offline forever after the initial registration; the deployer wallet doesn't need to know it.

The catch, again, is that the signature check happens in the witness layer, not the circuit. A hostile prover can hand the smart contract any (price, seq) pair it wants and the circuit just verifies that seq advanced, not that the pair was actually signed. The protection is real but partial. When Midnight's stdlib eventually ships an ecVerify primitive, this pattern hardens by a lot. Until then, think of it as Pattern 1 plus replay protection plus key separation. Good enough for most DeFi rate references. Probably not enough for anything with audit-grade requirements.

Pattern 3. Cross-contract.

Pattern 1 and Pattern 2 both assume your team owns the oracle. Pattern 3 is for the case where someone else's smart contract is already publishing the data you want. A protocol publishes an exchange rate. A governance smart contract publishes a parameter. You want your smart contract to read it.

Compact has no native cross-contract call instruction. A circuit cannot call another deployed smart contract during proof generation. What it can do is read the other smart contract's exported ledger state through the Midnight indexer, which has a complete view of public on-chain data, and then run that data through the circuit as a witness. The trick is enforcing that the witness can't lie about what state was actually current.

// pattern3-cross-contract.compact
pragma language_version >= 0.22;
import CompactStandardLibrary;

export ledger priceSourceId: Bytes<32>;
export ledger cachedPrice: Uint<64>;
export ledger cachedVersion: Uint<32>;

struct SourceSnapshot {
    price:   Uint<64>;
    version: Uint<32>;
}

// TypeScript reads the source contract's exported ledger via the indexer.
witness current_source_price(): SourceSnapshot;

export circuit initialize(sourceId: Bytes<32>): [] {
    priceSourceId = disclose(sourceId);
    return [];
}

export circuit sync_price(): [] {
    const snap = current_source_price();
    assert(disclose(snap.version) > cachedVersion, "Version must advance");
    cachedPrice   = disclose(snap.price);
    cachedVersion = disclose(snap.version);
    return [];
}

export circuit get_cached_price(): Uint<64> {
    return cachedPrice;
}
Enter fullscreen mode Exit fullscreen mode

The source smart contract is whatever Pattern 1 or Pattern 2 smart contract publishes a (price, version) pair where version increments on every update. The consumer caches the most recent (price, version) it has seen. The assert(disclose(snap.version) > cachedVersion, "Version must advance") line is the entire integrity story. A malicious TypeScript layer cannot feed an old snapshot into the consumer because the consumer remembers the version it last accepted and refuses anything that isn't strictly newer.

import { MidnightIndexerClient } from '@midnight-ntwrk/indexer-client';

const indexer = new MidnightIndexerClient(INDEXER_URL);
const PRICE_SOURCE_ADDRESS = '0xDeployedPriceSourceAddress...';

async function current_source_price(): Promise<{
  price: bigint;
  version: bigint;
}> {
  const state = await indexer.getLedgerState(PRICE_SOURCE_ADDRESS);

  return {
    price:   BigInt(state.price),
    version: BigInt(state.version),
  };
}

const tx = await consumerContract.callCircuit(
  'sync_price',
  {},
  { witnesses: { current_source_price } }
);
await tx.submit();
Enter fullscreen mode Exit fullscreen mode

There's an attack class the version check kills. Without it, you'd be relying purely on the TypeScript witness to be honest. A malicious prover could write a witness that returns last week's price, watch the consumer cache it, and then exploit downstream smart contracts that read the consumer's cached value. The version assertion makes that impossible. The only thing the malicious prover can do is refuse to call sync_price at all, which leaves the consumer with stale data but doesn't roll it backward.

Now the part the version check does NOT save you from. The price field itself. The TypeScript layer can still hand the consumer a snapshot like (version=6, price=999999) even when the source smart contract is actually at (version=5, price=100), and as long as 6 is greater than the cached version, the consumer will swallow it. Whether that matters depends entirely on what's running on the source side. A Pattern 2 source with signed updates and a real sequence guard means the witness can't fabricate version 6 cleanly without forging a signature it doesn't have. A Pattern 1 source with no signature at all means the version check is mostly decoration, you've just trusted the TypeScript layer to read honestly, which it has no obligation to do. Pattern 3 inherits the integrity story of whatever you point it at; it doesn't manufacture one.

So Pattern 3 in one breath: indexer is the anchor, the version counter kills rollback, the price field is exactly as honest as your source smart contract is. Reach for it when you're stacking on top of someone's already-signed feed. Don't reach for it when you're stacking on top of nothing, you'll just be wrapping their weakness in extra glue.

Property Pattern 1 — Admin Push Pattern 2 — Witness-Signed Pattern 3 — Cross-Contract
Trust anchor Whoever holds the admin key Oracle signing key (signature verified in TypeScript) Midnight indexer + source contract's trust model
Replay protection None on-chain; staleness checked client-side Sequence number — strictly increasing, enforced on-chain Version counter — strictly increasing, prevents rollback
Key compromise Attacker can post any value at any time Circuit only verifies seq advanced, not that payload was signed Inherits source contract's exposure exactly
Use it for Dashboards, prototypes, off-chain-supervised params DeFi rate references; signing key in KMS or HSM Composing on top of an already-signed Pattern 2 feed
Don't use it for Anything that holds value Audit-grade requirements (no on-chain sig verification yet) Stacking on top of an unsigned source — adds glue, not security

Picking one

Picking comes down to two questions and they're not symmetric. Who's writing the data, and what's downstream when it's wrong.

If you own the data and the surface is a prototype or an internal display, Pattern 1 is the answer and anything more is overengineering. The trust model matches the threat model, ship it and move on.

Crank the stakes up. Real money flowing through the consumer side, audit pressure, retail users. Now you want Pattern 2, with the signing key sitting in a KMS or HSM and not on somebody's laptop. The sequence guard is doing actual work there, not theater. Plan to move the signature check on-chain the day Compact's stdlib supports it.

Pattern 3 is the awkward one and the place people get this wrong. You only reach for it when the data lives in a smart contract someone else owns, which means the question stops being "which pattern do I want" and becomes "what does their smart contract look like under the hood." A Pattern 2 source means the composition is genuinely solid. Their signature, their sequence, your version cache, the integrity layers actually compound. A Pattern 1 source means you've now staked your smart contract's correctness on both their admin key and your own TypeScript witness layer, which is worse than trusting either one alone, and dressing it up with a version counter does not fix that. Worth checking before you wire anything together.

What's still missing

A few things would make the oracle story on Midnight a lot tighter. A native ecVerify circuit, which would let Pattern 2 do its signature check on-chain instead of in TypeScript glue. A canonical way to embed a Merkle proof of source smart contract state into a witness, so Pattern 3 could verify the snapshot it received actually came from the indexer's current view rather than just trusting the TypeScript fetch. A standard library of registry smart contracts where multiple oracle providers post signed feeds, so DApps can read from a multi-source view instead of one signing key.

None of those are here yet. The patterns above are what you have to work with in language version 0.22. They cover the obvious shape of the trust question, which is: you can't get data into a ZK circuit without trusting someone, the question is who and how the chain enforces what they're allowed to say.

That's it. Three smart contracts, one playground compiler that actually works without Docker, a couple compiler errors I had to chase down on the way. The code in this writeup compiles. The trust models are real. Pick the one that matches your threat model, not the most sophisticated one.

If something here is wrong I'd rather know now than after someone deploys it. Drop a note on the issue thread.

Related links


Submitted for Midnight Contributor Hub Bounty #304. Code compiles against compiler 0.30.0, language version 0.22. Payout in NIGHT after KYC.

Top comments (0)