Tokenize Before You Prompt: LLM Insights Without Exposing Data
Swap data for abstract tokens before calling an LLM, then detokenize the response. You get AI-powered insights while the real data never leaves your server.
Introduction
While working on LinkRail Analytics, I wanted to add AI powered insights to give site owners information about top pages, referrers, traffic trends, and funnel data. While any one URL is public, I wanted to focus on having as little as possible leave my servers when it comes to my customer’s data. LinkRail Analytics is highly focused on security and privacy.
I didn’t want to send any actual data through to the AI since all I care about is reporting on trends. I decided to create a system of abstract tokens that represent the pages/page titles/referrers/etc that I send to the AI and then decode them back so that the app’s dashboards could report with the actual titles/urls.
A quick word on “token,” since this is an AI article: I am not describing the tokens an LLM counts when it reads a prompt. I mean it in the data-security sense, a stand-in value that replaces something real, the same idea payment systems use when they swap a card number for a harmless surrogate.
I will be upfront about something: in my case most of this data is already public. A page title or a URL is not exactly a secret, and I tokenize it anyway, partly out of habit and partly because I would rather my customers’ data never leave my infrastructure at all, even when it is harmless. The exact same pattern is how you would handle data that genuinely is sensitive, and that is really the point of this article. The technique does not care whether the string is a public URL, an email buried in a support message, or location data; it only cares that the real value never leaves your server.
Requirements
When designing this system, I had some requirements for myself:
- Real data never leaves my servers - page titles, URLs, referrers, and anything else identifying gets swapped for an abstract token before the API call, so the third-party provider only ever sees tokens.
- The insights lose nothing - the model has to reason about the tokens exactly as it would the real values, and the dashboards still report with the actual titles and urls.
- Cheap to add - new insight types should only have to declare which fields to tokenize; the privacy boundary should come for free.
- Works on free text, not just structured fields - sometimes the sensitive value is not a field you control; it is woven into something a user typed, and the same pattern has to handle that too.
How the Token Swap Works
Page titles get replaced with PAGE_### and urls get replaced with URL_### as part of the payload with any statistics about those pages like visits, unique visitor counts, traffic over time, etc. When the analysis comes back, I can then replace the tokens with their actual titles/urls and persist the analysis in the database.
For example, a row of page data that starts out like this:
{ "page_title": "MLB on ESPN - Scores, Stats and Highlights", "url": "https://www.espn.com/mlb/", "visits": 1240 }gets rewritten to this before it ever leaves my server:
{ "page_title": "PAGE_1", "url": "URL_1", "visits": 1240 }The visit counts and other statistics pass through untouched, since those are just numbers, and a number on its own tells the AI nothing about who visited or what they read. The model reasons about PAGE_1 and URL_1 exactly as it would the real strings, and when the response comes back referring to PAGE_1, I swap the real title back in right before saving. The result is the same quality of insight with none of the identifying content ever touching a third-party API.
Writing the Prompts
Everything flows through two layers of prompting. A single system prompt sets the persona and the ground rules, and it stays the same for every request. On top of that, a per-analysis user prompt supplies the actual (tokenized) data and asks a focused question.
To start this project, I created a fixed system prompt. Its exact wording does not matter much here, because it carries no data at all, just instructions: use specific numbers, keep each insight to 2-3 sentences, format numbers consistently, and respond in plain text with no markdown so the result can drop straight onto a dashboard. It is identical on every request.
The second layer is the per-analysis user prompt, and this is the only place data enters. By the time it is built the data has already gone through the tokenizer, so what lands in the prompt is PAGE_# and URL_#, never real titles or links. The example below is stripped of its class boilerplate and instructions reduced for simplicitiy, the whole thing is just a template with the tokenized JSON dropped in:
public static function prompt(string $pages_json): string
{
return <<<PROMPT
Here are the top pages by visit count for a website analytics project:
{$pages_json}
Provide 2-3 insights about engagement, low-traffic pages, and content strategy.
PROMPT;
}That {$pages_json} is the only variable in the prompt, and it holds tokens rather than real data. Everything the model learns about those pages, it learns as PAGE_# and URL_#.
Base Insights Class
The following abstract class is extended in my application via ProjectInsights() and TeamInsights() so that each can leverage this abstract tokenization of data.
The HTTP call is inlined into callLlm() below (simplified for this article from a dedicated LlmClient() class) so that nothing here is a black box. The endpoint is OpenAI-compatible, so the same code works across providers. Notice that callLlm() is also where the system prompt from earlier gets attached to every request, right alongside the tokenized user prompt.
<?php
/**
* Tokenized LLM Insights
*
* Abstract base class for generating AI-powered insights.
* Sensitive data is replaced with abstract tokens before sending
* to the LLM, then restored in the response.
*
* Subclasses implement buildPrompts() and persist().
*/
abstract class Insights
{
private array $token_map = [];
private array $token_counters = [];
private array $quoted_prefixes = ['PAGE'];
/**
* Return an associative array of prompt key => user prompt string.
* Return null for any prompt that lacks sufficient data.
*
* @return array<string, string|null>
*/
abstract protected function buildPrompts(): array;
/**
* Persist the generated insights.
* (Save to your database however you like: ORM, raw SQL, etc.)
*
* @param array $insights
*/
abstract protected function persist(array $insights): void;
/**
* Orchestrates: build prompts → call LLM → detokenize → persist.
*/
public function generate(): array
{
$insights = [
'data' => [],
'updated_at' => date('Y-m-d H:i:s'),
];
foreach (array_filter($this->buildPrompts()) as $key => $user_prompt) {
$response = $this->callLlm($user_prompt);
if (!empty($response['content'])) {
$insights['data'][] = [
'type' => $key,
'content' => $this->detokenize($response['content']),
];
}
}
$this->persist($insights);
return $insights;
}
/**
* Send the system + tokenized user prompt to the LLM and return its text.
*
* The endpoint is OpenAI-compatible, so this same request shape
* works across providers.
*
* @param string $userPrompt The tokenized user prompt
* @return array{content: string}
*/
protected function callLlm(string $userPrompt): array
{
$payload = json_encode([
'model' => getenv('LLM_MODEL'),
'messages' => [
['role' => 'system', 'content' => SystemPrompt::prompt()],
['role' => 'user', 'content' => $userPrompt],
],
'temperature' => getenv('LLM_MODEL_TEMPERATURE'),
'max_tokens' => getenv('LLM_MODEL_MAX_TOKENS'),
]);
$ch = curl_init(getenv('LLM_ENDPOINT') . '/chat/completions');
curl_setopt_array($ch, [
CURLOPT_POST => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HTTPHEADER => [
'Content-Type: application/json',
'Authorization: Bearer ' . getenv('LLM_API_KEY'),
],
CURLOPT_POSTFIELDS => $payload,
CURLOPT_TIMEOUT => 30,
]);
$response = curl_exec($ch);
$http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
$errno = curl_errno($ch);
curl_close($ch);
// On any failure, return empty content so generate() simply skips this insight.
if ($errno || $http_code >= 400) {
return ['content' => ''];
}
$body = json_decode($response);
return ['content' => $body->choices[0]->message->content ?? ''];
}
/**
* Replace a real value with a token like URL_1, PAGE_3, REF_2.
* Deduplicates: the same value always gets the same token.
*/
protected function tokenize(string $prefix, string $value): string
{
$existing = array_search($value, $this->token_map, true);
if ($existing !== false) {
return $existing;
}
$this->token_counters[$prefix] ??= 0;
$this->token_counters[$prefix]++;
$token = $prefix . '_' . $this->token_counters[$prefix];
$this->token_map[$token] = $value;
return $token;
}
/**
* Batch-tokenize fields on an array of rows.
*
* @param array $rows Data rows (objects or arrays)
* @param array $fields Field name => token prefix, e.g. ['url' => 'URL', 'title' => 'PAGE']
*/
protected function tokenizeRows(array $rows, array $fields): array
{
return array_map(function ($row) use ($fields) {
foreach ($fields as $field => $prefix) {
if (is_object($row) && !empty($row->$field)) {
$row->$field = $this->tokenize($prefix, $row->$field);
} elseif (is_array($row) && !empty($row[$field])) {
$row[$field] = $this->tokenize($prefix, $row[$field]);
}
}
return $row;
}, $rows);
}
/**
* Restore real values in the LLM response text.
* Sorts longest-first to prevent partial token matches.
* Tokens with quoted prefixes get wrapped in double quotes.
*/
private function detokenize(string $text): string
{
$tokens = array_keys($this->token_map);
usort($tokens, fn($a, $b) => strlen($b) - strlen($a));
foreach ($tokens as $token) {
$prefix = substr($token, 0, strrpos($token, '_'));
$value = $this->token_map[$token];
if (in_array($prefix, $this->quoted_prefixes, true)) {
$value = '"' . $value . '"';
}
$text = str_replace($token, $value, $text);
}
return $text;
}
}A few things worth calling out in the base class:
tokenize()is the core swap. It hands back a stable token for a value and dedupes as it goes, so the same URL always maps to the sameURL_#. That consistency matters, because it keeps the token map small and lets the model recognize that two rows refer to the same page.tokenizeRows()is just a convenience wrapper for the common case: run a batch of database rows throughtokenize(), mapping each sensitive field to a prefix (['page_title' => 'PAGE', 'url' => 'URL']).detokenize()reverses everything on the way out. Two details keep it correct: it replaces the longest tokens first soURL_1never partially clobbersURL_10, and it wraps certain prefixes (like page titles) in quotes so the restored text reads naturally when a human-readable title lands mid-sentence.
Because the token map only lives in memory for the duration of a single generate() run, there is no persistent mapping of real data to tokens sitting around anywhere either.
Example Child Class
Here is an implementation of an analytics project leveraging the base insights class. This is where the tokens actually get created: buildTopPages() pulls page titles and URLs from the database and runs them through tokenizeRows() before handing them to the prompt. For contrast I have also included buildTrafficTrends(), which skips tokenization entirely because it only deals in counts. Not every insight needs it, and the base class lets each method opt in.
<?php
/**
* LLM Project Insights Service
*
* Generates AI-powered insights for a single project.
*/
namespace LinkRail\Services\Llm;
use LinkRail\Models\{
Project,
ProjectPageSummary
};
use LinkRail\Services\Llm\Prompts\{
TopPagesPrompt,
TrafficTrendsPrompt
};
use LinkRail\Types\AiInsightTypes;
class ProjectInsights extends Insights
{
private int $project_id;
public function __construct(int $project_id)
{
$this->project_id = $project_id;
}
protected function buildPrompts(): array
{
return [
AiInsightTypes::traffic_trends->value => $this->buildTrafficTrends(),
AiInsightTypes::top_pages->value => $this->buildTopPages(),
// add more insight types here
];
}
protected function persist(array $insights): void
{
$db = new Project();
$db->values([
['insights', json_encode($insights)]
]);
$db->where([
['id', '=', $this->project_id]
]);
$db->update();
}
private function buildTrafficTrends(): ?string
{
$db = new Project();
$db->select('visits, unique_visitors, visits_per_day');
$project = $db->find($this->project_id);
if (empty($project->visits_per_day)) {
return null;
}
return TrafficTrendsPrompt::prompt(
$project->visits_per_day,
(int) $project->visits,
(int) $project->unique_visitors
);
}
private function buildTopPages(): ?string
{
$db = new ProjectPageSummary();
$db->select('page_title, url, visits');
$db->where([
['project_id', '=', $this->project_id]
]);
$pages = $db->orderby('visits', 'desc')->limit(20)->findAll();
if (count($pages) === 0) {
return null;
}
// Swap the real titles and URLs for tokens before the data ever leaves the server.
$pages = $this->tokenizeRows($pages->toArray(), [
'page_title' => 'PAGE',
'url' => 'URL',
]);
return TopPagesPrompt::prompt(json_encode($pages));
}
}That is the whole trick in one method: query the sensitive rows, hand the field-to-prefix map to tokenizeRows(), and encode the result. From here on, the prompt, the API call, and the raw response all deal exclusively in PAGE_# and URL_#. The base class stitches the real values back in during detokenize() before anything is persisted.
Adding a new insight type is the same three steps with a different prefix. My referrers analysis tokenizes the referrer field into REF_#, and my funnel analysis tokenizes segment names into SEG_#, and each one is just another buildPrompts() entry pointing at its own tokenizeRows() map. The privacy boundary comes for free every time.
Implementation
So now that we have the base and child classes defined, we can move onto implementation. For LinkRail, I have a worker running in Amazon Web Services (AWS) Elastic Container Service (ECS) that is hooked up to AWS Simple Queue Service (SQS). AWS EventBridge triggers a message to the queue on a schedule that triggers the worker to create the AI powered insights.
// snippet from my worker. i have more where conditions in my app,
// but for the sake of this article, "get me all projects" works.
// I have also trimmed the exception just to show that there is
// an opportunity for logging and sending exception data to a service like Sentry.
//.....
$db = new Project();
$db->select('id');
$projects = $db->withCount()->findAll();
if ($projects->foundRows() > 0) {
foreach ($projects as $project) {
try {
$service = new ProjectInsights($project->id);
$service->generate();
} catch (InsightException $e) {
logger(LogChannels::sqs)->error('[AI INSIGHTS]', [ 'body' => [ 'message' => $e->getMessage(), 'project_id' => $project->id ] ]);
Sentry\captureException($e);
}
}
}The worker loops every project, builds and runs its insights, and moves on. Because tokenization is baked into the base class, the worker never has to think about privacy at all; it just calls generate(). Each project’s prompts are tokenized, sent, detokenized, and persisted in the database.
When You Can’t Just Use an ID
Before anything else: the best version of this is to never send PII at all. If a row belongs to a user, send their user_id and join whatever human details you need back on your side once the response comes back from the AI. Tokenizing a field that already maps to a user you can identify is usually a sign you skipped the identifier you had. Tokenization is not permission to get lazy about that.
Let’s say you want to do analysis on support tickets, contact form message, surveys, etc. These are cases where you genuinely cannot use a user ID as the identifying information lives inside content your users wrote, not in a column you control. There is no key to swap in, because an email or phone number is woven into a sentence the model actually needs to read.
Say you want the model to triage incoming support messages to find where customers mention payment issues to create reports or dashboards, and one of them looks like this:
My account jane@example.com was double charged this morning and I need
someone to call me back at 555-123-4567 today.You cannot replace that with a user ID. You may have a support desk user id, but you still need to strip out the PII. The point here is to read the message. But you can still tokenize the identifiers inside it before it leaves your server. The tokenize() primitive from earlier does not care whether the value came from a database column or a regex match:
private function tokenizeText(string $text): string
{
// Emails and phone numbers are reliably detectable with a pattern.
$patterns = [
'EMAIL' => '/[\w.+-]+@[\w-]+\.[\w.-]+/',
'PHONE' => '/\b\d{3}[.\-\s]?\d{3}[.\-\s]?\d{4}\b/',
];
foreach ($patterns as $prefix => $pattern) {
$text = preg_replace_callback(
$pattern,
fn($match) => $this->tokenize($prefix, $match[0]),
$text
);
}
return $text;
}Now the model sees this instead:
My account EMAIL_1 was double charged this morning and I need
someone to call me back at PHONE_1 today.It can still classify the ticket as a billing problem and tag it urgent all without the real address or phone number ever being part of the request. detokenize() restores them in the response exactly like it does for page titles.
Two honest caveats. Structured identifiers like emails, phone numbers, and card numbers are easy to catch with a pattern, but names are not. Reliably finding “Jane Doe” in free text is a named-entity-recognition problem, not a regex, which is a good reason to keep names in structured fields whenever you can, rather than fishing them out of free-text after the fact. TLDR; be careful with any data that you send through to an LLM.
Conclusion
The pattern comes down to a simple boundary: real data lives on my side of the wire, tokens live on the AI’s side. Looking back at the requirements I set out at the start:
- Real data never leaves my servers - The third-party API only ever sees the tokens. Whether it is
PAGE_1,URL_1, orEMAIL_1, even a logged or retained request holds nothing that identifies a customer’s pages, visitors, or users. - The insights lose nothing - Trends, spikes, drop-offs, and funnel analysis are all about relationships between numbers, and those survive tokenization.
- Cheap to add - New insight types just declare which fields to tokenize; the base class handles the rest.
- Works on free text - The same
tokenize()primitive that swaps database fields can be setup to handle identifiers woven into user-written content, with a regex pass standing in for the field map.
If you are reaching for an LLM but hesitating because of where your data would end up, tokenizing before you prompt is a small amount of code that lets you have both.
✌️ Matt