Illustrative design example. This is not a verified factual account. This article exists to demonstrate the blog's layout and editorial structure. Its narrative and figures have not been fact-checked for publication.
235,365 procurement processes. 48,055 verified awards. 54.7 million annual procurement-plan rows. 390,763 profiled bidders. The numbers are large enough to look authoritative before anyone asks whether the rows describe the same thing.
They do not.
TL;DR
Public records become a product only after the system keeps source documents, normalized facts, and derived claims separate. A query result is not evidence by itself. It needs provenance, a time boundary, and a way to show what the source did not say.
Four counts, four different meanings
The procurement corpus behind Octograma contains several layers. Each layer answers a different question.
| Layer | Recorded scale | Safe interpretation |
|---|---|---|
| Procurement processes | 235,365 | Processes available in the working corpus |
| Verified awards | 48,055 | Outcomes linked to evidence under the current verification rules |
| Procurement-plan rows | 54.7M | Planning records, not completed contracts |
| Profiled bidders | 390,763 | Entities resolved well enough for the current profile model |
Putting the four figures in one sentence creates an illusion of continuity. There is no honest funnel from 54.7 million plan rows to 48,055 awards without qualification. Plans, processes, awards, and bidder identities have different update cycles and failure modes.
The first engineering decision was therefore not a model. It was a boundary.
Preserve the source before improving it
Normalization makes records searchable. It can also erase the reason a result looks strange.
A changed company name, a duplicated process identifier, an amended award, or a malformed date may become invisible after a clean transformation. The normalized table looks better precisely when the source needs more attention.
The system keeps three layers:
- Source: the document or record as received, with retrieval time and origin.
- Fact: a normalized field with the transformation that produced it.
- Inference: a score, grouping, or recommendation that depends on facts and assumptions.
The layers can point forward. They must also point back. If a profile cannot reach the record that produced it, the interface has created confidence without evidence.
Missing data is not a negative result
Government data often arrives unevenly. A field can be empty because the buyer omitted it, the source changed its schema, the record is still preliminary, or the ingestion failed.
Those cases should not collapse into “no.”
This matters most in product features that rank opportunities or estimate a bidder's fit. A missing award history does not mean a bidder has never won. An unpublished document does not mean the process lacks the condition. An empty query does not prove an empty market.
Octograma keeps win probability inactive and labeled as in validation. That is not a launch delay hidden behind legal copy. It is the product state implied by the evidence.
The interface carries part of the proof
Data rigor can disappear at the last mile. A model can preserve provenance internally while the UI turns it into a confident badge.
The interface needs to expose at least four things:
- the source of the claim
- the time of the latest observation
- the transformation or rule applied
- the difference between missing, preliminary, and verified
This adds friction. It also prevents a recommendation from becoming stronger in presentation than it is in the underlying record.
What changed in the build
The practical rules are plain:
- Counts display their snapshot date.
- Derived claims never overwrite source facts.
- Preliminary and verified outcomes use different states.
- Every ranking feature defines what an empty value means.
- Features stay inactive when the evidence boundary is not ready.
None of these rules makes the model more sophisticated. They make the system more difficult to misread.
Assumptions & caveats
- The figures are corpus snapshots: they change as ingestion, deduplication, and verification continue.
- Verified means verified under a rule: it does not mean a legal determination or a complete view of every procurement event.
- Entity resolution is probabilistic: names, identifiers, and corporate changes can split one bidder or merge distinct ones.
- A planning row is not an award: procurement plans describe intent and can change before a process exists.
The dangerous public-data product is not the one with an obvious error. It is the one that removes every visible sign of uncertainty while the source remains incomplete.