Building a RAG-enabled ChatBot using AWS Bedrock, Lex, and Kendra

Sign upSign inSign upSign inMatthew VenneFollowITNEXT--ListenShareIn case you haven’t heard, GenAI is the bees knees. When used properly, it can accelerate work product tremendously. Most folks are aware of OpenAI ChatGPT and Google Bard, but what about building your own custom ChatBot to accelerate your work? AWS offers several service offers that accelerate AI development. Today we’re going to be examining these services and I integrated them into our solution.Amazon Lex is a service for building chatbots using voice and text that leverages the same conversation engine as Amazon Alexa. It natively integrates with Slack, FaceBook Messenger, Kik, and Twilio SMS. It even offers a visual IDE for desiging your chatbots — similar to Step Functions Workflow Studio.A high level of how it works:Amazon Kendra is an AI-driven enterprise search service that enables searching across different content repositories with built-in connectors. S3, SharePoint, Box, Confluence, Jira, SalesForce are just a few of locations with built-in support.However, you can also use Amazon Kendra for Retrieval Augmented Generation — to enrich the chat bot’s performance.Retrieval Augmented Generation (RAG) is one means of incorporating custom data into a foundational model. This requires no re-training of the models; instead, it takes a vector representation of the user prompt and passes it to a database that has data mapped according to this vectorization — This is how Kendra can enrich a prompt before passing it to the LLM. The other options for enriching a foundational model’s output is through fine-tuning which requires re-training a model with labeled data stored in S3 — this requires Provisioned Throughput.RAG is typically quicker to implement but slower in performance since it must query the external database every time. Fine-tuning typically requires more time to implement but is more more performant. It is possible to incorporate both techniques in a single AI system.Note: Bedrock does have limited support for Continued Pre-training using unlabeled data but this is in Preview and only supported on the Titan Text Express model.Amazon Bedrock is a serverless, AI solution that offers streamlined access to many leading LLMs for testing — Anthropic, Amazon, AI21 Labs, and Meta are a few providers. It supports fine-tuning to customize your models and also R.A.G. with the following methods:My company asked me to research some Generative AI solutions that we could leverage to accelerate our work product. I wanted to have a POC in place quickly so I could show the capabilities to our leadership by end of the week. Most of the services I used were brand new to me — conceptually all of the AWS Services made sense but learning the nuances of them is always the hard part.The first step was to ingest the data into Kendra using one of their connectors. I found this blog by AWS that outlined the process.aws.amazon.comThe next step was step was to build the Chatbox. I found this AWS Sample to get me started.github.comA few things before running the template:I deployed the template and was ready to move to the next step — the slack integration. Lex supports OAuth for authenticating Slack and other external systems.slack.comI tested it out and after about 10 seconds it returned a response!----ITNEXTMatthew is Managing Director at stackArmor — a leading cloud security and compliance partner. All opinions expressed are his own. . stackarmor.comHelpStatusAboutCareersBlogPrivacyTermsText to speechTeams