← Natawat Damrongsilp · Portfolio

Case study · System architecture

Immigration Officer AI Assistant

A bilingual chat assistant that lets immigration officers ask for statistics and for handbook procedures in plain Thai or English, and get a sourced answer in seconds instead of waiting days for a database specialist or paging through a 300-page manual.

The problem

Officers ask two very different kinds of question

One kind is answered by a database, the other by a printed handbook. Before this project, neither had a fast path. The assistant had to handle both from the same chat box, decide which kind it was given, and never invent an answer for an agency where a wrong number or a wrong procedure has real consequences.

“How many…?” questions

10 อันดับสัญชาติที่เดินทางเข้าประเทศไทยมากที่สุดในปี 2568

Top 10 nationalities entering Thailand in 2568 (Buddhist Era, = 2025)

Before: a request to a database specialist, answered in days.

“What do I do when…?” questions

ผู้โดยสารเปลี่ยนเครื่องที่มีหมายจับของประเทศไทย เจ้าหน้าที่ต้องดำเนินการอย่างไร

A transit passenger has a Thai arrest warrant. What must the officer do?

Before: searching a printed handbook of several hundred pages while the passenger waits.

Architecture

One chat box, three answer paths, one model that only writes

Everything except the language model runs on a single CPU-only virtual machine. Deterministic rules decide how a question is answered; the model is called last, to turn retrieved facts into a readable answer, and it streams back to the browser token by token.

System architecture of the Immigration Officer AI Assistant An officer's browser talks to Nginx, which proxies to a FastAPI backend. The backend normalises the question, detects its language, and a rule-based router sends it down one of three paths: statistics via reviewed SQL templates on a PostgreSQL warehouse, handbook retrieval via a ChromaDB vector index, or questions about an uploaded document via a per-session index. An answer composer sends the retrieved facts to a Llama 4 Scout model served by vLLM on a remote GPU cluster, checks citations, and streams the answer back. Redis holds conversation context and rate limits; Prometheus and Grafana monitor the system. APPLICATION SERVER · ONE CPU-ONLY VM REMOTE GPU CLUSTER reached over a secure tunnel Officer React web app Thai / English HTTPS Nginx serves the app proxies /api FastAPI backend Python · one container Guard & normalise rate limit · common misspellings · abbreviations Language + context answer in the question's language · follow-ups Intent router deterministic rules, no model involved rules pick one path one path per question Statistics 44 reviewed SQL templates · ≤ 15 rows out Handbook (RAG) semantic search · relevance gate · citations Your document the officer's upload · this session only Answer composer the model writes only from retrieved facts Citation check + stream invented references flagged · sent as SSE streamed answer · chart · sources App database PostgreSQL 16 · users, audit audit Redis context · rate limits context Stats warehouse PostgreSQL 16 SQL Handbook index ChromaDB · bge-m3 search Session index expires after 30 min search prompt + retrieved facts tokens vLLM inference Llama 4 Scout 17B-16E Instruct · FP8 H100 GPUs metrics Prometheus scrapes every 15 s · 5 alert rules Grafana live dashboards

Scroll sideways to see the whole diagram.

One request through the system. The router picks exactly one path; the model receives only what that path retrieved, and its answer is checked and streamed back. Colours mark the three paths throughout this page.

Request lifecycle

What happens between Enter and the first word

  1. Guard

    Rate limit per client, then fix common misspellings, Thai numerals and abbreviations such as ตม. → ตรวจคนเข้าเมือง.

  2. Record

    Write an audit entry for the question before anything is answered.

  3. Detect language

    If more than a quarter of the characters are Thai, the answer is in Thai; otherwise English.

  4. Recall context

    A short-lived note of the last question (30 minutes) lets “make it a pie chart” or “what about China?” inherit the earlier query.

  5. Route

    Rules choose statistics, handbook or the uploaded document. An attached document goes first.

  6. Retrieve

    Run one reviewed SQL template, or search the handbook index, or search the session's document index.

  7. Compose

    The model writes from the retrieved rows or passages only, and must cite them.

  8. Check & stream

    Citations are checked against what was retrieved, then text, chart and source chips stream to the browser.

Routing

How the router decides

Routing is plain, inspectable rules rather than a model call. A misrouted question becomes one rule change and one new test case, and the same question always takes the same path.

CHECK 1Is a document attached, and does the question match it well enough?
Document

Answer strictly from the file's own text.

CHECK 2Does it look statistical, like “10 อันดับ…”, “how many”, a year or a nationality?
Statistics

Fill one of 44 templates. Years in the Buddhist Era and the Thai fiscal year are understood, and synonyms such as พม่า/เมียนมา map to one nationality. If it turns out not to be statistical after all, it falls through to the handbook.

CHECK 3Everything else: a procedure, a rule, a scenario.
Handbook

Semantic search over the handbook; complex scenarios are split into topics and searched topic by topic.

CHECK 4Is the best handbook match below the relevance threshold?
Decline

Say the handbook does not cover it, instead of guessing.

Design decisions

The choices that shaped it

The model never writes SQL

Statistics come from 44 hand-written, reviewed query templates. The model only fills in a narrative around at most 15 result rows.

Rejected: free-form text-to-SQL. For a security agency a plausible but wrong query is worse than no answer, and generated SQL is hard to audit.

Rules route, the model writes

Choosing the answer path is deterministic. The language model appears in exactly one place: composing the answer from what was retrieved.

Rejected: an LLM router. Cheaper to run and easier to debug as rules, and every routing decision can be explained.

Silence beats a confident guess

The handbook path only answers when the best passage reaches a relevance score of 0.42, and passages scoring below 0.35 never reach the prompt.

Rejected: always answering. An officer acting on an invented procedure is the failure that matters most.

Every procedure is traceable to a page

Answers cite section, item and page of the handbook, with a link to that page of the handbook PDF. A check after generation flags any citation the retrieved passages do not support.

Why: officers trust the answer when they can verify it in one click.

No GPU on the application server

The app runs on one modest CPU-only VM. Inference runs on a shared GPU cluster, reached through a secure tunnel.

Trade-off: question text leaves the app server for the inference cluster; the next phase moves the model fully on-premise.

Uploads are short-lived

An uploaded PDF, Word, Excel or CSV file is indexed for that conversation only, and the index expires after 30 minutes.

Why: officers can query a working document without adding it to the shared knowledge base.

Answer formats

The shape of the answer follows the situation

Routine question · 3 parts

Short procedural questions.

  1. Direct answer
  2. Steps
  3. Handbook references

Complex scenario · 5 parts

Long questions, or two or more immigration terms (visa, overstay, warrant, passport…). Written for an officer who needs to know what to do first.

  1. Immediate action
  2. Procedure, step by step
  3. Who has authority to decide
  4. Legal basis
  5. Exceptions the handbook mentions

Statistics answers add a chart chosen from six types (bar, horizontal bar, pie, line, grouped bar, dual line), an expandable data table, and Copy, CSV and PDF export.

Evaluation

How it was tested

TestScopeResult
Handbook Q&A benchmark120 questions built from the procedures handbook with immigration-law reviewers. The 15 declines are questions the handbook does not cover, answered with “not in the handbook”; the one fail had a stray foreign word in the answer. 104 pass · 15 declined · 1 fail
Statistics benchmark65 questions from the agency's technology centre. Of the 63 passes, 25 were answered with a chart and table and 38 were correctly declined because the requested period was not in the warehouse. 63 pass · 2 review · 0 fail
System integration testEnd-to-end cases on the deployed system16 / 16
User acceptance testAcceptance scenarios on the deployed system12 / 12

Stack

Built with

Interface

  • React single-page app
  • Server-sent events for streaming
  • Nginx

Application

  • Python · FastAPI · Uvicorn
  • JWT sessions, role-based admin views
  • Docker

Data

  • PostgreSQL 16
  • Redis
  • ChromaDB

Retrieval

  • BAAI/bge-m3 embeddings (multilingual)
  • Cosine similarity, relevance gating
  • Topic-split retrieval for scenarios

Model

  • Llama 4 Scout 17B-16E Instruct (FP8)
  • vLLM, OpenAI-compatible API
  • H100 GPU cluster

Operations

  • Prometheus · Grafana
  • Audit trail of logins, questions and uploads
  • Five alert rules on errors, latency, retrieval quality, feedback and review queue

Lessons

What the project taught

The dangerous failures are the silent ones. A renamed metadata field in the vector index once removed every citation without raising a single error. The reader now accepts both field layouts, and the evaluation checks citations explicitly rather than only the answer text.

Score relevance against the officer's own words. The decline rule is measured against the original question, not the internally rewritten search query, so “not in the handbook” reflects what was actually asked.

Determinism is a feature for government users. Rule-based routing and fixed query templates make every answer explainable and every fix testable, which suits an organisation that has to account for each answer.