Open to offers
Rodrigo
Torres
AI Engineer
- SUNAT, payment gateways and ledgers
- I design the architecture and build it
- years of software
- 6
- years of AI in production
- 2
- systems of my own
- 4
- systems in production
- 20+
Two years with models in production. Six years building software.
BM25 + vector search // Reciprocal Rank Fusion // MCP servers // FastAPI // Step Functions // AWS Lambda // CDK // DynamoDB // walk-forward // PostgreSQL // electronic invoicing // pytest
A search written for this page: 39 passages, BM25 plus vectors fused with RRF. It is not PrecioVivo, which is the case study and runs over a price catalogue.
I wrote a gold set and the fusion lost one
The box above was described as hybrid and graded by nobody. Twenty-six questions, each with the passages that genuinely answer it, marked by reading the passages rather than by reading what the search returned. Three configurations differing in exactly one thing each.
- Conditions
- n = 26 · k = 4
- Pasajes
- 39
- Preguntas
- 14 léxicas · 12 semánticas
- Embeddings
- gemini-embedding-001 · 256d
- Medido
- 2026-08-28
Vectores de consulta cacheados en disco, para que dos corridas no difieran por el servicio.
| Configuration | Recall@4 | MRR | NDCG@4 |
|---|---|---|---|
| Lexical only (BM25) | 0.577[0.39, 0.74] | 0.563 | 0.489 |
| Vector only | 0.846[0.66, 0.94] | 0.790 | 0.745 |
| Fused with RRF● | 0.769[0.58, 0.89] | 0.682 | 0.634 |
The lexical half alone misses eleven of twenty-six, and they are the questions that share no word with their passage: whether he builds alone, whether he has broken production, what salary he is after. That is precisely what a vector index exists to answer.
Fusion recovers a question neither half finds on its own, which is the evidence that fusing is not decoration: two weak signals in different lists add up to a position neither reached alone.
It pays for that. The fusion's MRR sits below the vector half's: it puts the answer inside the top four more often and leaves it slightly lower within those four. With four passages going to the model that costs little; with one result on screen it would cost a great deal.
In English the fusion comes out below the vector half alone, 0.769 against 0.846, losing all three questions where they disagree. At n = 26 the sign test gives p = 0.25, which does not separate that from chance. Switching configuration per language on that evidence would be fitting the system to the gold set, so both run the same one.
The Wilson interval is the part that cannot be skipped. A recall of 0.923 over 26 cases is 0.923 [0.76, 0.98]: the system could be at 0.76 and this gold set would not know. Twenty-six cases are a starting point, not an answer.
A real defect came out of this. The two lists entered the fusion at different depths: the lexical one whole, the vector one cut at 12. BM25 keeps every passage scoring above zero, a median of 31 out of 39, so a passage ranked 30th still collected a vote that no comparable vector match could answer. Cutting both to the same depth took Spanish recall from 0.885 to 0.923. The correction is justified by the argument; the figure only says what it had been costing.
Hiring an AI engineer means hiring numbers nobody can verify.
Almost every AI portfolio claims results without saying how they were measured or where the system fails. I publish the measurement conditions next to every figure, and a whole section devoted to what breaks. If I cannot link it to its source, it does not appear.
How I work ↗Closeness is the measure
A vector index does not store words, it stores directions. Every fragment is a point, and answering a query means asking which points face the same way. Move the cursor closer and watch what each stroke does: that is what happens 9,281 times when you type a question.
The field answers the cursor. The index answers the query.
Every figure with the conditions it was taken under
Recall of the search isolated from the deterministic floor, with the real embedder. The toy embedder gave 0.759: that difference is the only thing proving the semantic half is not decoration
With the real embedder. Asking for papaya, which is not in the catalogue, returns potato records: 0.6444 against 0.6656, two hundredths apart. No threshold separates them without killing the aggregate questionsunsolved in retrieval
MCP server in production for the RAG system
The same surface, exposed over HTTP
435 in pytest, 77 in vitestcoverage missing
Same walk-forward validation. Lost to AR(1)name the metric
Limits carry the same weight as results.
Papaya returns potato, and I did not fix it in retrieval
Asking for a product that is not in the catalogue returns the closest one by name. Papaya scores 0.6444 and potato 0.6656: two hundredths apart. I tried the obvious threshold and it does not work, because any cut that separates them kills the aggregate questions, which score 0.5151. I solved it by warning the model from the lexical filter, which already knew no word matched the catalogue. The evaluation still reports both violations: I did not relax the gold set. ↗
It breaks at 510 products
514 bytes per Map element against the 256 KB Step Functions state ceiling. Past that number the Map has to be paginated or the payload moved to S3. ↗
Shall we talk?
I answer quickly, and never with a template.