Integrating Chatbots In Ecommerce

Last week, I shared how Gen AI is moving us from the age of information to the age of intelligence. Technology is changing rapidly and the way customers shop and buy is changing, too. We need to understand how the customer journey is evolving in order to drive customer connection today. That is our bread and butter at HubSpot - we’re deeply curious about customer behavior! So I want to share one important shift we’re seeing and what go-to-market teams can do to adapt. Traditionally, when a customer wants to learn more about your product or service, what have they done? They go to your website and explore. They click on different pages, filter for information that’s relevant to them, and sort through pages to find what they need. But today, even if your website is user-friendly and beautiful, all that clicking is becoming too much work. We now live in the era of ChatGPT, where customers can find exactly what they need without ever having to leave a simple chat box. Plus, they can use natural language to easily have a conversation. It's no surprise that 55% of businesses predict that by 2024, most people will turn to chatbots over search engines for answers (HubSpot Research). That’s why now, when customers land on your website, they don’t want to click, filter, and sort. They want to have an easy, 1:1, helpful conversation. That means as customers consider new products they are moving from clicks to conversations. So, what should you do? It's time to embrace bots. To get started, experiment with a marketing bot for your website. Train your bot on all of your website content and whitepapers so it can quickly answer questions about products, pricing, and case studies—specific to your customer's needs. At HubSpot, we introduced a Gen AI-powered chatbot to our website earlier this year and the results have been promising: 78% of chatters' questions have been fully answered by our bot, and these customers have higher satisfaction scores. Once you have your marketing bot in place, consider adding a support bot. The goal is to answer repetitive questions and connect customers with knowledge base content automatically. A bot will not only free up your support reps to focus on more complex problems, but it will delight your customers to get fast, personalized help. In the age of AI, customers don’t want to convert on your website, they want to converse with you. How has your GTM team experimented with chatbots? What are you learning? #ConversationalAI #HubSpot #HubSpotAI

The real challenge in AI today isn’t just building an agent—it’s scaling it reliably in production. An AI agent that works in a demo often breaks when handling large, real-world workloads. Why? Because scaling requires a layered architecture with multiple interdependent components. Here’s a breakdown of the 8 essential building blocks for scalable AI agents: 𝟭. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸𝘀 Frameworks like LangGraph (scalable task graphs), CrewAI (role-based agents), and Autogen (multi-agent workflows) provide the backbone for orchestrating complex tasks. ADK and LlamaIndex help stitch together knowledge and actions. 𝟮. 𝗧𝗼𝗼𝗹 𝗜𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗶𝗼𝗻 Agents don’t operate in isolation. They must plug into the real world:  • Third-party APIs for search, code, databases.  • OpenAI Functions & Tool Calling for structured execution.  • MCP (Model Context Protocol) for chaining tools consistently. 𝟯. 𝗠𝗲𝗺𝗼𝗿𝘆 𝗦𝘆𝘀𝘁𝗲𝗺𝘀 Memory is what turns a chatbot into an evolving agent.  • Short-term memory: Zep, MemGPT.  • Long-term memory: Vector DBs (Pinecone, Weaviate), Letta.  • Hybrid memory: Combined recall + contextual reasoning.  • This ensures agents “remember” past interactions while scaling across sessions. 𝟰. 𝗥𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸𝘀 Raw LLM outputs aren’t enough. Reasoning structures enable planning and self-correction:  • ReAct (reason + act)  • Reflexion (self-feedback)  • Plan-and-Solve / Tree of Thought These frameworks help agents adapt to dynamic tasks instead of producing static responses. 𝟱. 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗕𝗮𝘀𝗲 Scalable agents need a grounding knowledge system:  • Vector DBs: Pinecone, Weaviate.  • Knowledge Graphs: Neo4j.  • Hybrid search models that blend semantic retrieval with structured reasoning. 𝟲. 𝗘𝘅𝗲𝗰𝘂𝘁𝗶𝗼𝗻 𝗘𝗻𝗴𝗶𝗻𝗲 This is the “operations layer” of an agent:  • Task control, retries, async ops.  • Latency optimization and parallel execution.  • Scaling and monitoring with platforms like Helicone. 𝟳. 𝗠𝗼𝗻𝗶𝘁𝗼𝗿𝗶𝗻𝗴 & 𝗚𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲 No enterprise system is complete without observability:  • Langfuse, Helicone for token tracking, error monitoring, and usage analytics.  • Permissions, filters, and compliance to meet enterprise-grade requirements. 𝟴. 𝗗𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁 & 𝗜𝗻𝘁𝗲𝗿𝗳𝗮𝗰𝗲𝘀 Agents must meet users where they work:  • Interfaces: Chat UI, Slack, dashboards.  • Cloud-native deployment: Docker + Kubernetes for resilience and scalability. Takeaway: Scaling AI agents is not about picking the “best LLM.” It’s about assembling the right stack of frameworks, memory, governance, and deployment pipelines—each acting as a building block in a larger system. As enterprises adopt agentic AI, the winners will be those who build with scalability in mind from day one. Question for you: When you think about scaling AI agents in your org, which area feels like the hardest gap—Memory Systems, Governance, or Execution Engines?

🤖 How To Design Better AI Experiences. With practical guidelines on how to add AI when it can help users, and avoid it when it doesn’t ↓ Many articles discuss AI capabilities, yet most of the time the issue is that these capabilities either feel like a patch for a broken experience, or they don't meet user needs at all. Good AI experiences start like every good digital product by understanding user needs first. 🚫 AI isn’t helpful if it doesn’t match existing user needs. 🤔 AI chatbots are slow, often expose underlying UX debt. ✅ First, we revisit key user journeys for key user segments. ✅ We examine slowdowns, pain points, repetition, errors. ✅ We track accuracy, failure rates, frustrations, drop-offs. ✅ We also study critical success moments that users rely on. ✅ Next, we ideate how AI features can support these needs. ↳ e.g. Estimate, Compare, Discover, Identify, Generate, Act. ✅ Bring data scientists, engineers, PMs to review/prioritize. 🤔 High accuracy > 90% is hard to achieve and rarely viable. ✅ Design input UX, output UX, refinement UX, failure UX. ✅ Add prompt presets/templates to speed up interaction. ✅ Embed new AI features into existing workflows/journeys. ✅ Pre-test if customers understand and use new features. ✅ Test accuracy + success rates for users (before/after). As designers, we often set unrealistic expectations of what AI can deliver. AI can’t magically resolve accumulated UX debt or fix broken information architecture. If anything, it visibly amplifies existing inconsistencies, fragile user flows and poor metadata. Many AI features that we envision simply can’t be built as they require near-perfect AI performance to be useful in real-world scenarios. AI can’t be as reliable as software usually should be, so most AI products don’t make it to the market. They solve the wrong problem, and do so unreliably. As a result, AI features often feel like a crutch for an utterly broken product. AI chatbots impose the burden of properly articulating intent and refining queries to end customers. And we often focus so much on AI that we almost intentionally avoid much-needed human review out of the loop. Good AI-products start by understanding user needs, and sparkling a bit of AI where it helps people — recover from errors, reduce repetition, avoid mistakes, auto-correct imported files, auto-fill data, find insights. AI features shouldn’t feel disconnected from the actual user flow. Perhaps the best AI in 2025 is “quiet” — without any sparkles or chatbots. It just sits behind a humble button or runs in the background, doing the tedious job that users had to slowly do in the past. It shines when it fixes actual problems that it has, not when it screams for attention that it doesn’t deserve. Useful resources: AI Design Patterns, by Emily Campbell https://www.shapeof.ai AI Product-Market-Fit Gap, by Arvind Narayanan, Sayash Kapoor https://lnkd.in/duEja695 [continues in comments ↓]

I am deploying my own LLM Mistral-7B-instruct with supercharged inference As I work on building a chat assistant with Mistral-7B to help customers navigate complex SAAS platform, I run into an important consideration, how will I scale and serve the LLM running the assistant. Let's look at a scenario: Using one GPU-A100 for deployment, our LLM Mistral-7B can generate 17 tokens per second. Now, lets say, if we have 1000 customers using our assistant at the same time, and average length of response from assistant is 150 tokens, putting the numbers together, our assistant will take 2 hours to process requests at anytime. An average reader's speed is 240 words per minute which we should match so our readers don't get bored but with the above setup, more than half the customers could even be waiting 1 hour to get any text at all. Not good at all for User Experience!! First, lets define the metrics we will use to assess the performance of LLM in the context of deployment: - Latency : Total time taken to process one user query. Important for better UX - Throughput: The number of tokens generated per second by the system. Important for scalability We are going to use a popular framework vLLM for optimization and benchmarking but lets look at the basic principles that vLLM leverages: 1. KV caching: - Transformer decoder architecture generates tokens sequentially and to generate a token, it uses all the past generated tokens. For each new token, a key-value vectors are generated which measures the relevance of the token to previous tokens. - So lets say, if we want to predict xth token, we will need KV vectors for 1...(x-1)th tokens, these vectors can be cached instead of regenerating them for every token, leading to time optimization with a memory trade-off. 2. Continuous batching our main optimization: - We parallelly process batches of customer queries, enhancing throughput. - However, differing response sizes in generative text lead to inefficient GPU memory use. - For examples: lets create a batch of two queries: - 'Delhi is the capital of which country?' -'Tell me about Harry potter' The first requires a brief response, while the second could be lengthy. With equal memory allocation per query, the GPU waits for the longer response to complete, leading to underutilized memory for the shorter query. This results in a hold-up of memory resources that could have been used for processing other queries. vLLM allows the efficient use of GPU memory to cache KV vectors, such that when a query in a batch is finished, another query can start processing in that batch. Observations on using vLLM on a batch of 60 queries: 1. Latency decreased more than 15x with vLLM 2. Throughput increased from 18 tokens/s to 385 tk/s 3. Throughput shows significant boost on large batches Link to reproduce results on colab: https://lnkd.in/ew_S_2WD If you are working on a similar project, you are welcome to share your experience :)

As an analyst, I was intrigued to read an article about Instacart's innovative "Ask Instacart" feature integrating chatbots and chatgpt, allowing customers to create and refine shopping lists by asking questions like, 'What is a healthy lunch option for my kids?' Ask Instacart then provides potential options based on user's past buying habits and provides recipes and a shopping list once users have selected the option they want to try! This tool not only provides a personalized shopping experience but also offers a gold mine of customer insights that can inform various aspects of a business strategy. Here's what I inferred as an analyst : 1️⃣ Customer Preferences Uncovered: By analyzing the questions and options selected, we can understand what products, recipes, and meal ideas resonate with different customer segments, enabling better product assortment and personalized marketing. 2️⃣ Personalization Opportunities: The tool leverages past buying habits to make recommendations, presenting opportunities to tailor the shopping experience based on individual preferences. 3️⃣ Trend Identification: Tracking the types of questions and preferences expressed through the tool can help identify emerging trends in areas like healthy eating, dietary restrictions, or cuisine preferences, allowing businesses to stay ahead of the curve. 4️⃣ Shopping List Insights: Analyzing the generated shopping lists can reveal common item combinations, complementary products, and opportunities for bundle deals or cross-selling recommendations. 5️⃣ Recipe and Meal Planning: The tool's integration with recipes and meal planning provides valuable insights into customers' cooking habits, preferred ingredients, and meal types, informing content creation and potential partnerships. The "Ask Instacart" tool is a prime example of how innovative technologies can not only enhance the customer experience but also generate valuable data-driven insights that can drive strategic business decisions. A great way to extract meaningful insights from such data sources and translate them into actionable strategies that create value for customers and businesses alike. Article to refer : https://lnkd.in/gAW4A2db #DataAnalytics #CustomerInsights #Innovation #ECommerce #GroceryRetail

𝗧𝗼𝗼 𝗺𝗮𝗻𝘆 𝗯𝗿𝗮𝗻𝗱𝘀 𝘁𝗿𝗲𝗮𝘁 𝗪𝗵𝗮𝘁𝘀𝗔𝗽𝗽 𝗹𝗶𝗸𝗲 𝗦𝗠𝗦. 𝗜𝘁’𝘀 𝗻𝗼𝘁. That's a criminal misuse of WhatsApp that’s quietly killing retention for both D2C and B2B brands. Brands get access to the WhatsApp API, upload a list, and hit “Send to All.” It feels efficient. But it creates what we call the broadcast trap, a pattern that burns through customer trust fast. 𝗪𝗵𝘆 𝗶𝘁 𝗗𝗼𝗲𝘀𝗻’𝘁 𝗪𝗼𝗿𝗸: Without enough personalization, messages feel generic and irrelevant. Customers start ignoring future messages after 1–2 interactions. Engagement and repeat purchase rates drop significantly. We’ve seen this across hundreds of brands before they changed their strategy to: → 𝗖𝗼𝗻𝘁𝗲𝘅𝘁𝘂𝗮𝗹 𝘁𝗮𝗿𝗴𝗲𝘁𝗶𝗻𝗴: Messages are sent based on user actions, such as abandoned carts, product views, or purchase inactivity. → 𝗦𝗲𝗴𝗺𝗲𝗻𝘁-𝘀𝗽𝗲𝗰𝗶𝗳𝗶𝗰 𝗰𝗼𝗺𝗺𝘂𝗻𝗶𝗰𝗮𝘁𝗶𝗼𝗻: Returning customers, first-timers, and high-LTV buyers each get a different experience. → 𝗧𝗶𝗺𝗲𝗹𝘆 𝘁𝗿𝗶𝗴𝗴𝗲𝗿𝘀: Instead of one big push, messages are sent at the right moment — like 2 hours after a missed checkout, or 1 day before an offer expires. → 𝗣𝗿𝗼𝗴𝗿𝗲𝘀𝘀𝗶𝘃𝗲 𝗰𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻𝘀: Each interaction builds on the last instead of restarting from scratch → 𝗖𝗹𝗲𝗮𝗿 𝗼𝗽𝘁-𝗶𝗻𝘀 𝗮𝗻𝗱 𝗽𝗮𝗰𝗶𝗻𝗴: Customers feel in control, not spammed. → 𝟮 -𝘄𝗮𝘆 𝗰𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻𝘀: Hooking each message with contextual chatbots that continue the conversation. 1-way announcements don’t work, 2-way chats do. Here’s what changes when the 𝗰𝗼𝗻𝘃𝗲𝗿𝘀𝗮𝘁𝗶𝗼𝗻 𝗯𝗲𝗰𝗼𝗺𝗲𝘀 𝘁𝗵𝗲 𝘀𝘁𝗿𝗮𝘁𝗲𝗴𝘆: Higher conversion rates Better repeat purchase rates Dramatically fewer unsubscribes and spam reports That’s the power of doing WhatsApp 𝘳𝘪𝘨𝘩𝘵. And for those wondering how brands manage this kind of personalization at scale? They use tools that make it effortless (we built one we’re pretty proud of 😉).

𝐑𝐨𝐚𝐝𝐦𝐚𝐩 𝐟𝐨𝐫 𝐁𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐒𝐜𝐚𝐥𝐚𝐛𝐥𝐞 𝐀𝐈 𝐀𝐠𝐞𝐧𝐭𝐬 Building AI Agents That Scale Isn’t Just About LLMs — It’s About Architecture. If you’re just plugging a model into a chatbot, you’re missing the bigger picture. This 7-step roadmap lays out how to design scalable, capable AI agents that go beyond simple prompt-response patterns — moving toward autonomy, memory, and collaboration: 1️⃣ 𝑷𝒊𝒄𝒌 𝒂𝒏 𝑳𝑳𝑴 Choose models that support reasoning and stability. (LLaMA, Claude, Mistral are great starting points.) 2️⃣ 𝑩𝒖𝒊𝒍𝒅 𝑨𝒈𝒆𝒏𝒕'𝒔 𝑳𝒐𝒈𝒊𝒄 Define how your agent should think: Should it reflect before responding? Plan actions? Use tools? 3️⃣ 𝑾𝒓𝒊𝒕𝒆 𝒊𝒕𝒔 𝑪𝒍𝒆𝒂𝒓 𝑶𝒑𝒆𝒓𝒂𝒕𝒊𝒏𝒈 𝑰𝒏𝒔𝒕𝒓𝒖𝒄𝒕𝒊𝒐𝒏𝒔 Craft reusable templates for consistency, especially when interfacing with APIs or tools. 4️⃣ 𝑨𝒅𝒅 𝑴𝒆𝒎𝒐𝒓𝒚 Use sliding window techniques for short-term recall, and vector databases for long-term memory (ZepAI, MemGPT). 5️⃣ 𝑪𝒐𝒏𝒏𝒆𝒄𝒕 𝑻𝒐𝒐𝒍𝒔 & 𝑨𝑷𝑰𝒔 Agents shouldn’t just talk — they should act. Connect them to search, databases, CRMs, etc. 6️⃣ 𝑮𝒊𝒗𝒆 𝑰𝒕 𝒂 𝑱𝒐𝒃 "Be helpful" isn't enough. Narrow scopes like “summarize insights” or “respond with markdown” improve outcomes. 7️⃣ 𝑺𝒄𝒂𝒍𝒆 𝒕𝒐 𝑴𝒖𝒍𝒕𝒊-𝑨𝒈𝒆𝒏𝒕 𝑻𝒆𝒂𝒎𝒔 Distribute responsibilities: one agent for planning, another for execution, a third for QA. 📌 Pro Tip: Use task-specific naming conventions and orchestration frameworks like LangGraph or CrewAI to manage your agent network efficiently. 🔖 Save this as a quick reference! 𝑾𝒂𝒏𝒕 𝒕𝒐 𝒄𝒐𝒏𝒏𝒆𝒄𝒕 𝒘𝒊𝒕𝒉 𝒎𝒆? 𝘍𝒊𝒏𝒅 𝒎𝒆 𝒉𝒆𝒓𝒆 --> https://lnkd.in/dTK-FtG3 Follow Shreya Khandelwal for more such content. ************************************************************************ #LargeLanguageModels #ArtificialIntelligence #GenerativeAI #LLM #MachineLearning #AI #DataScience #AIagents #AgenticAI #LangChain #MultiAgent #PromptEngineering #OpenAI

📱 Rakuten’s launch of its AI Agent across mobile and web platforms is an example of how Asia Pacific retailers are moving from reactive conversational assistants to more proactive, ecosystem-driven capabilities. 👉 While Rakuten’s AI Agent is not yet “agentic AI” as per IDC’s definition, it reflects the starting point of a wider journey: adopting assistants for customer support and product discovery, then progressing toward autonomous, intent-driven agents that coordinate pricing, fulfillment, and engagement in real time. 📊 IDC’s Industry Insights 2025 – Retail Survey shows this evolution is underway: > 61% of retailers globally are piloting and deploying agentic AI > Nearly 25% of initiatives are already live > In Asia Pacific, adoption is especially visible in customer service, product discovery, and omni-channel integration 🔎 Along this path, retailers in APAC face key priorities: > Integration and Data Infrastructure – Building cloud-native and unified commerce platforms to support real-time decisioning > Personalization and Local Preferences – Balancing predictive analytics with community-driven, local brand trust > Operational Efficiency – Piloting AI in dynamic pricing, fraud detection, and workforce support, while addressing ROI and privacy concerns > Omni-Channel Intelligence – Moving from channel coordination to seamless, data-driven fulfillment and engagement > Conversational Interfaces – Preparing for customer expectations around voice, chat, and image search, which can lower friction and even reduce returns 🌏 Beyond Rakuten, we see JD.com in China experimenting with AI shopping concierges, Shopee piloting conversational agents in Southeast Asia, and Lotte in Korea exploring autonomous pricing and merchandising systems. Each step brings retail closer to the agentic ecosystems IDC describes. 📌 These developments reflect IDC’s finding that personalization, omni-channel integration, and agentic AI are converging to reshape retail operations and customer engagement. 📖 Read more in the full report by IDC's Ornella Urso: Personalized CX and Omni-Channel Integration Evolving Rapidly Alongside Agentic AI (Jul 2025 – https://lnkd.in/gSMq6eum) #RetailAI #AsiaPacific #AgenticEcosystem #CustomerExperience #IDCInsights See more about Rakuten here: https://lnkd.in/gPikKi8M