# How to et "Enterprise RAG Experience" Without the Enterprise Job

Open any AI engineer or Forward Deployed Engineer job post and you’ll see some version of the same line: **“RAG experience on real enterprise data.”**

That leaves most people stuck. How do you get enterprise experience without already working at an enterprise?

This week’s answer is a dataset I keep coming back to, plus the lab I built around it.

![](https://substackcdn.com/image/fetch/$s_!3OJl!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F419d0a33-4e90-447e-873d-384675a0aaca_1456x816.png)

## Meet TechQA

TechQA is a **free dataset of real IBM technical support questions**, asked by real IBM customers. It isn’t synthetic and it isn’t tidy Wikipedia trivia.

![](https://substackcdn.com/image/fetch/$s_!Dszt!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F275f4816-315f-4a15-a1be-76c201133cca_1247x1112.png)

![](https://substackcdn.com/image/fetch/$s_!JyIK!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdff96b13-c74c-4022-9a40-fc5c7c90ea8a_1672x941.png)

Every question comes with **50 candidate documents** (IBM “technotes”):

- **One** is the right page.
- **The other 49 are look-alikes**: same product, same error codes, almost the same title.
- And **sometimes none of them are right**.

That last point is the important one. It’s exactly what a client’s knowledge base looks like: messy pages that all sound the same, and questions where the honest answer is *“I can’t find support for that.”*

Learn to handle that and you’ve learned what clients actually pay for.

## The lesson you can’t fake

Here’s what separates people who’ve *built* this from people who’ve *watched videos* about it.

TechQA tells you **where each answer sits inside its source page** (a start and end offset). So you can measure something most tutorials never check.

When I measured it, **16% of answers start after character 2,000.**

Now look at what a lot of tutorials do:

```
text = doc[:2000]   # “just keep the opening of each document”
```

For that 16%, the answer is **gone before search even begins**. No retriever, re-ranker or prompt can recover text you’ve already thrown away.

You only learn this by measuring.

## What you’ll build in the RAG Lab

![The RAG Lab home page: a left sidebar listing the course modules and a main panel with the title, an intro, and cards for each level](https://substackcdn.com/image/fetch/$s_!OXbB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F57bc95f3-a26a-4c17-b7d3-29f0c54b1877_1200x675.webp)

The lab takes you from raw data to a **measured baseline**:

1. **Inspect the data**: splits, answerable vs. unanswerable questions, candidate lists.
2. **Compare raw HTML with clean text**: what does stripping actually change?
3. **Measure answer positions**: where do answers live, and what does truncation cost you?
4. **Build a BM25 search**: a simple, honest keyword baseline.

On **160 real questions**, the BM25 baseline found:

- **the right page first 47% of the time**
- **the right page in the top 5 63% of the time**

**That’s your number to beat.** Even better, the failures tell you what to fix next: **the query, the documents, or the ranking**.

That’s the exact workflow you’d walk an interviewer through: data first, then measurement, then a baseline, then targeted fixes.

## Real scale, on your laptop

You start with about **28,500 documents**, then move to the full library of **802,000**. That’s 28× bigger.

At that size you hit the problems you’d normally only meet on the job: indexing time, memory, and whether your results still hold up as the data grows.

## Try this before next week

Even if you never open the lab, try this one exercise on whatever RAG project you have:

> ***Find out where your answers live.** For a handful of questions you know the answers to, record the character position of the answer in its source document. Then check whether your chunking or truncation keeps it.*

If you find answers past your cut-off, you’ve just found a bug that no amount of prompt engineering will fix.

## What’s next

This is **Part 1 of a 3-part series on RAG for FDEs**: retrieval, or finding the right source *before* you ask a model to answer.

You don’t need an enterprise job to build enterprise skills. You need the right dataset, honest measurement, and a system that knows when to say *“I can’t find support for that.”*

%[https://www.youtube.com/watch?v=74K4UN4M7cw]

Hit reply and tell me your BM25 numbers (or where your answers ended up living). I read every reply.

See you in Part 2, Zahir *The Adaptive Engineer*
