Privacy-first AI shopping assistant with visual search & voice agent
Bibian — Multilingual AI E-Commerce Platform
01 / Problem
Cross-border e-commerce support for platforms like Bibian Taiwan faces high operational costs and complexity when hiring multilingual agents across 4 languages (English, Japanese, Traditional & Simplified Chinese). Human customer service is slow to scale for repetitive order status and return inquiries, while off-the-shelf cloud AI chatbots frequently provide irrelevant responses and pose privacy risks by transmitting customer data to external processors. Furthermore, traditional keyword-only search fails when shoppers search with photos or vague queries, and shoppers get overwhelmed parsing hundreds of unorganized product reviews.
02 / Approach
Engineered Bibian AI Assistant to solve these cross-border challenges through a multi-modal AI layer. A custom Rasa NLU model trained on anonymized 4-language chat logs automatically resolves domain-specific FAQs and order status queries over encrypted REST APIs without exposing customer data to third-party clouds. OpenAI CLIP vector embeddings power visual photo search so shoppers can upload product images to find matching items, while Deepgram WebSocket voice integration enables real-time speech shopping with a lip-synced video avatar. Finally, LLM integrations automate customer review summaries, product Q&A, and SEO description generation for store admins.
03 / Architecture
Next.js 15 frontend with 4-language i18n proxied to an Express 4 backend and MongoDB Atlas. An encrypted local Rasa NLU pipeline resolves order intents securely, a Flask + CLIP microservice manages visual photo search, Socket.io syncs live carts, and Groq/Ollama LLMs power review summaries and admin description tools.
04 / Outcome
Delivered a fast, secure, and budget-friendly AI platform that slashes support costs, eliminates cloud privacy leaks, speeds up order inquiries, and delivers a complete visual and voice shopping experience across 4 languages.