Led an agent and RAG assistant that supports online customer consultation and recommends products.
01Context
- Samsung Electronics wanted an assistant that answers customer questions consistently across offline stores, online stores, and consultation agents.
Problem
- Customers researched reviews and prices on external sites and came to Samsung channels only to buy.
- Staff could not answer comparison and promotion questions on the spot, and delivery and repair inquiries stalled on phone transfers, costing sales.
- Each step of the customer journey, from discovery to repurchase, was answered by a different party.
My role
- Held overall project responsibility and led architecture design and core module development in a team of about 30.
- Defined the platform architecture, 18 response scenarios, data model, and admin requirements, and built the orchestrator pipeline, predefined answer and guardrail modules, and prompt templates hands-on.
Approach
- Built a channel-independent platform behind an API gateway, so new channels and features reuse the same modules.
- Evolved the architecture over three versions, from a single-pipeline MVP to separate structured and unstructured retrieval with reranking and guardrails, then to parallel execution.
- In the final version, split queries into sub-queries, ran structured, unstructured, and web search in parallel, and branched simple and informative questions.
- Established an evaluation framework for the RAG service and automated evaluation with Ragas.
Key decisions
| Decision | Why | Trade-off |
|---|
| A modular platform, not a single chatbot | Modules are reused when channels or features are added, and an admin page lets the operations team take over. | Slower initial MVP delivery. |
| Entity mapping and pseudo queries instead of direct Text-to-SQL | Extracted entities are mapped to codes and a predefined view schema, so queries can be validated and stay secure. | Mapping tables must be maintained for each new query type. |
| Predefined answers that bypass the LLM | Greetings, FAQs, and policy-restricted answers come from a table, cutting cost and latency. | Less flexible answers and a table to maintain. |
Architecture
| Layer | Components | Flows |
|---|
| 01 Channels | - Offline stores
- Online stores
- Agent console
| →to 02 API gateway |
| 02 API gateway | - Channel-independent interface
| →to 03 Orchestrator |
| 03 Orchestrator | - Query rewrite
- Intent triage
- Entity extraction, code mapping, and pseudo query
- Predefined answers and guardrails
- Merge and response layout
| →to 04 Retrieval and storage · Parallel queries→to 05 LLM |
| 04 Retrieval and storage | - Structured data (PostgreSQL)
- Unstructured data (Azure AI Search)
- Web search
- Logs and cache
| |
| 05 LLM | | |
| 06 Admin | - Logs, monitoring, permissions, and sensitive-data controls
| ↩back to 03 Orchestrator |
Outcome
- Automated answers for 18 customer scenarios previously handled one by one by agents, and handed the service over to operations for online and offline stores.
- Halved average response time from 10 s to 5 s between versions 2 and 3.
- Cut LLM call cost by 50% by serving frequent questions with predefined answers.