Harnessing Natural Language for MongoDB Queries With Google Gemini
Venkatesh Shanbhag, Yemi Falokun7 min read • Published Apr 09, 2024 • Updated Apr 23, 2024
Rate this article
In the digital age, leveraging natural language for database queries represents a leap toward more intuitive data management. Vertex AI Extensions, currently in private preview, help in interacting with MongoDB using natural language. This tutorial introduces an approach that combines Google Gemini's advanced natural language processing with MongoDB, facilitated by Vertex AI Extensions. These extensions address key limitations of large language models (LLMs) by enabling real-time data querying and modification, which LLMs cannot do due to their static knowledge base post-training. By integrating MongoDB Atlas with Vertex AI Extensions, we offer a solution that enhances the accessibility and usability of the database.
MongoDB's dynamic schema, scalability, and comprehensive querying capabilities render it exemplary for Generative AI applications. It is adept at handling the versatile and unpredictable nature of data that these applications generate and use. From personalized content generation, where user data shapes content in real time, to sophisticated, AI-driven recommendation systems leveraging up-to-the-minute data for tailored suggestions, MongoDB stands out. Furthermore, it excels in complex data analysis, allowing AI tools to interact with vast and varied datasets to extract meaningful insights, showcasing its pivotal role in enhancing the efficiency and effectiveness of Generative AI applications.
Natural language querying represents a paradigm shift in data interaction, allowing users to retrieve information without the need for custom query languages. By integrating MongoDB with a system capable of understanding and processing natural language, we streamline database operations, making them more accessible to non-technical users.
The solution involves a synergy of several components, including MongoDB, the Google Vertex AI SDK, Google Secrets Manager, and OpenAPI 3 specifications. Together, these elements create a robust framework that translates natural language queries into MongoDB Data API calls. In this solution, we have explored basic CRUD operations with Vertex AI Extensions. We are closely working with Google to enable vector search aggregations in the near future.
- MongoDB: A versatile, document-oriented database that stores data in JSON-like formats, making it highly adaptable to various data types and structures
- Google Vertex AI SDK: Facilitates the creation and management of AI and machine learning models, including the custom extension for Google Vertex AI
- Vertex AI Extensions: Enhance LLMs by allowing them to interact with external systems in real-time, extending their capabilities beyond static knowledge
- Google Secrets Manager: Securely stores sensitive information, such as MongoDB API keys, ensuring the solution's security and integrity
- OpenAPI 3 Specification for MongoDB Data API: Defines a standard, language-agnostic interface to MongoDB that allows for both easy integration and clear documentation of the API's capabilities
The solution operates by converting natural language queries into parameters that the MongoDB Data API can understand. This conversion is facilitated by a custom extension developed using the Google Vertex AI extension SDK, which is then integrated with Gemini 1.0 Pro. The extension leverages OpenAPI 3 specifications to interact with MongoDB, retrieving data based on the user's natural language input. Google Secrets Manager plays a critical role in securely managing API keys required for MongoDB access, ensuring the solution's security.
Before you start, make sure you have the following:
- If you are new to MongoDB Atlas, you can sign up to MongoDB either through the Google Cloud Marketplace or with the Atlas registration page.
- Vertex AI Extensions are not publicly available. Please sign up for the Extensions Trusted Tester Program.
Before we begin, also make sure you:
Enable MongoDB Data API: To enable the Data API from the Atlas console landing page, open the Data API section from the side pane, enable the Data API, and copy the URL Endpoint as shown below.
To create an API key, click on Users and then click on Create API Key.
Create a secret for your API key: To secure the MongoDB Data API key, store it in Google Cloud [Secret Manager](https://cloud.google.com/secret-manager/docs/creating-and-accessing-secrets#:~:text=Go to the Secret Manager page in the Google Cloud console.&text=On the Secret Manager page,example%2C my-secret )). To create a new secret on the Google Cloud Console, navigate to Secrets Manager, and click on CREATE SECRET. Paste the secret created from MongoDB to the secret value field and click on Create.
Develop API specification: We will define the interactions between Google Vertex AI and your MongoDB Data API using OpenAPI 3.0 specs. You can download the specs from the GitHub repository. This specification outlines how natural language queries will be translated into MongoDB operations.
This tutorial uses the MongoDB default dataset from the sample_mflix database, movies collection. We will run all the below code on the Enterprise Colab notebook.
- Vertex AI Extensions is a platform for creating and managing extensions that connect large language models to external systems via APIs. These external systems can provide LLMs with real-time data and perform data processing actions on their behalf.
- Install the required Python dependencies.
- Once the dependencies are installed, restart the kernel.
- Initialize the environment variables.
- To create the Vertex AI extension, run the below script. The manifest here is a structured JSON object containing several key components:
- Validate the Created Extension, and print the Operation Schema and Parameters.
Once the extension is created, navigate to Vertex AI UI and then Vertex UI Extension on the left pane.
All the created extensions will be available for selection. Select MongoDB Vertex API Interpreter. For this tutorial, we will consider two user journeys.
Retrieve data through Vertex AI extensions: Imagine needing to quickly find out when a classic film was released without navigating through the database manually. By asking Vertex AI, "Find the release year of the movie 'A Corner in Wheat' from VertexAI-POC cluster, sample_mflix, movies," you get the specific release year, as the system performs a findOne() operation to retrieve this detail.
In addition to using the UI, we can also use the Notebook to query MongoDB using natural language processing. We are using Gemini to convert natural language to the actual operation schema for calling the Vertex AI extension. Update the environment variable and run the below script.
From the converted schema, query the extension.
Retrieve data through Vertex AI extensions: A film historian wants a list of all movies released in a specific year — say 1924 — to study the cinematic trends of that era. They could ask, "Give me movies released in the year 1924 from VertexAI-POC cluster, sample_mflix, movies," and the system would use the find() method to list all movies from 1924, providing a comprehensive snapshot of that year's cinematic output.
In addition to using the UI, we can also use the Notebook to query MongoDB using natural language processing.
Here’s the Gemini integration for natural language to extension schema conversion:
Execute the extension to get the response.
In conclusion, integrating MongoDB with natural language querying capabilities revolutionizes data interaction, enhancing accessibility and intuitiveness for database queries. Leveraging the Google Gemini Foundation Model alongside a custom Vertex AI extension not only enriches the data retrieval experience but also upholds data security and integrity. We are closely working with the Google Cloud team to add even more query patterns to Vertex AI extensions. Watch out for more in this space.
- Get started with MongoDB Atlas on Google Cloud.