PublicCVE

Live at publiccve.com — one corpus, plain-language search, and signals layered on top of the raw NVD dump.

Most CVE sites assume you’re starting from an identifier. You have CVE-2024-3400, or you heard “Log4Shell” on a podcast, and you want the page for it. That workflow is fine. It’s also not the one I kept getting stuck on.

The harder question is directional: given this attack text, which vulnerability does it look like? A honeypot line. A snippet from an incident report. A rule that didn’t fire because the encoding was new. Signature matchers answer that until they don’t. And if you bolt an LLM on top and let it free-form a CVE ID, a hallucination isn’t a typo — it’s a bad lead for an analyst.

I built PublicCVE to sit in that gap. It’s a CVE intelligence product I own end to end: ingest the public feeds into Postgres, search them in plain language (including through an MCP tool for agents), and run an attack→CVE matching path when the input is telemetry instead of an ID. I’ve sold it to companies; they integrate over simple APIs without hosting a few hundred thousand vectors themselves.

This is how I got there.

Three feeds, three shapes, one hub

NVD arrives as nested JSON. CISA’s Known Exploited Vulnerabilities catalog is flatter. EPSS is flatter still. The product UI needs one stable shape that isn’t married to any of those wire formats.

So I treat validation as three boundaries around one hub in Postgres. On the way in, each feed gets parsed into its own typed validation models — deserialize and coerce at the edge, map into the canonical CVE record, bulk-upsert where I need speed. On the way out, the HTTP API serializes through a separate response schema built from the ORM rows. The ingestion models and the API models never import each other. They only meet in the database.

That mattered the first time a KEV row landed before NVD did. Partial records had to be legal, not exceptional. Fields stay nullable. When two sources disagree on the same column, I track which source won.

The detail page is where you see why the merge work was worth it. A single CVE might carry a CVSS breakdown, KEV and ransomware flags, exploit/PoC badges, affected product versions parsed out of CPE strings, and a timeline of when each signal arrived.

One record, many feeds — KEV, EPSS, CVSS, CPE applicability, trending — merged with per-field provenance instead of “last writer wins.”

NVD’s CPE applicability data is especially annoying: it’s buried in a nested configurations tree, and NVD’s own CPE enrichment cratered after early 2024, so a vendor browse that only trusts one source quietly goes blank on new CVEs. The ingester pulls applicability from multiple sources and normalizes vendors and products into shared tables so “Apache” doesn’t get duplicated four hundred times under slightly different strings.

Embeddings on a schedule, not inline

Search needs vectors. I did not want ingestion calling an embedder on every upsert. If the model server is down, feeds should still land. Re-running ingestion shouldn’t re-embed three hundred thousand unchanged rows.

Ingestion marks a row stale when the text that feeds search actually changed. A separate backfill worker pulls stale rows in batches, builds a richer document than the raw NVD blurb — KEV title, vendor, product, CWE tokens folded in — embeds with Voyage, writes the vector, commits the batch, moves on. Ingestion pushes; embedding pulls. Eventual consistency on purpose.

I designed for interruption because the first full backfill got interrupted. Once on a context-length edge case where a token-dense CVE blew the model’s limit. Once when the machine ran out of memory and took Postgres with it. Committing per batch meant I picked up where I left off with no rework. Boring batch job. Boring saved me.

When semantic search buries FortiOS

Pure vector search is good at meaning and bad at exact tokens. Ask about FortiOS or CWE-502 or deserialization and the embedding can rank a fuzzy neighbor above the CVE you actually wanted. Keyword search has the opposite failure mode.

I run both over the same filtered candidate set — KEV, severity, EPSS, CWE, vendor, product — so both rankers see the same pool. Vectors through pgvector. Keywords through Postgres full-text search. Then I merge the ranked lists with Reciprocal Rank Fusion.

Semantic search misses exact tokens like product names and version strings; keyword search misses meaning. RRF combines on rank, not score, so I don’t have to normalize two incomparable relevance scales. An item that’s only mediocre semantically but a strong keyword hit still surfaces. I have a unit test that encodes exactly that claim, and a CI gate that seeds a tiny corpus with a deliberately wrong query vector so hybrid search has to beat pure-vector or the build fails. The API and the eval share one ranking path — I’m not testing a fork that could drift.

The query is prose, not a CVE ID. Filters shrink the pool first; hybrid retrieval ranks what’s left.

I also didn’t want “hybrid is better” to live only in my head. There’s a labeled golden set, MRR and Recall@k, and a live-corpus eval I run when the full embedding stack is up. Before I’d swap embedding models, that harness has to show retrieval didn’t get worse.

Attack text isn’t a search query

Matching an attack is a different question than search. You’re not typing “PAN-OS command injection” — you’re handing over raw telemetry. Log lines don’t sit next to CVE descriptions in embedding space.

HyDE is the bridge: rewrite the payload into CVE-style prose, embed that, run the same hybrid retrieval. For the final step the model doesn’t get to invent a CVE ID. It picks from what retrieval returned, or says no match. Wrong ID isn’t a prompt problem; it’s ruled out by the response shape. Analyses get logged with the evidence attached.

That’s the line I care about in security AI: the model can help rank and explain, but it can’t introduce facts that weren’t in the retrieval set.

News that points back at CVEs

Ingestion also pulls security news and links articles to the CVEs they reference, so the product isn’t only a static catalog. The newsroom is sorted by trending or latest; each card is coverage you can trace back to the underlying vulnerability record.

Reporting layered on top of the corpus — useful for “what’s hot right now,” not just “what’s CVE-2024-XXXX.”

One database, sold as a product

I kept vectors and relational data in one Postgres instead of standing up Elasticsearch beside it. I’d rather not pay a permanent sync-and-drift tax unless the workload actually outgrows the database — and I wrote down what “outgrows” would mean so I can’t hand-wave it later.

Customers plug in with text in, ranked CVEs or a grounded verdict out. They keep their own orchestration; I own the corpus, embeddings, and matching.

Where it landed

The site is live at publiccve.com with a few hundred thousand CVEs tracked, hybrid search, attack matching, browse and trending, API keys, and an MCP tool so agents search the same surface humans do. About 240 automated tests cover the critical paths, including the retrieval eval gate. I’ve sold it to companies integrating it into their security stacks. Saved-search alerts are partly built — the core retrieval and matching story is the part I’m proud of.

What it taught me

A handful of things I expect to carry forward: validate at the boundaries and let the database be the hub; decouple ingestion from embedding so feeds never block on a model server; fuse heterogeneous rankers on rank when their scores aren’t comparable; measure retrieval instead of asserting it; and when an LLM is in a high-stakes loop, constrain what it’s allowed to output instead of writing a longer prompt and hoping.

PublicCVE is the project where those lessons stopped being blog-post abstractions and became load-bearing design calls.