Deploy and Monitor a Mongo Replica Set
What is a Mongo Replica Set?
A replica set in MongoDB is a group of mongod processes that maintain the same data set. Replica sets provide redundancy and high availability, and are the basis for all production deployments.
Source: MongoDB Manual
About this Tutorial
In addition to ensuring your data layer is redundant and highly available, monitoring the data layer is key to production readiness. There are a number of ways to monitor your Mongo replica set in Railway, but this tutorial covers how to do so using popular open source tools, Prometheus and Grafana.
Objectives
In this tutorial, you will learn how to -
- Deploy a Mongo Replica Set from a Railway template
- Deploy a Python FastAPI app that connects to the replica
- Set up monitoring of the replica set with Prometheus and Grafana
Prerequisites
To be successful using this tutorial, you should already have -
- A Railway API token
- A Github account connected to Railway
- Latest version of Railway CLI installed
Most of the source code you will deploy as part of this tutorial can be found here.
By the end of this tutorial, you will have a project similar to the one below.
Let's get started!
1. Deploy the Mongo Replica Set
This step assumes you have created a Railway API token.
First, we will deploy the Mongo replica set using the template in the Railway marketplace.
- Click here to open the deployment page for the Mongo replica set
- Click
Configure
next to the init-mongo-ha service and add your Railway API token - Click
Deploy
After clicking Deploy, you will be taken to your new project in Railway.
Note: The init-mongo-ha service is designed to be ephemeral, existing only to initiate the Mongo replica set before it is removed.
2. Deploy the Python FastAPI app
Now we'll deploy a Python FastAPI and connect it to the Mongo replica set.
Create the app locally
From your local machine -
- Create a folder for your FastAPI app called
fastApi
- Within the
fastApi
folder, create the following 3 files -- main.py:
- Procfile:
- requirements.txt:
Create and Configure an Empty Service in Railway
Now that you have the necessary files to run the Python FastAPI app in Railway, let's deploy it.
In the Railway project that contains your Mongo replica set -
- Add a New service by clicking
+ New
- Select
Empty Service
- Add the following variables to the service
- In the service Settings, rename the service to
fastApi
- Click
Deploy
to apply and create the empty service - In the Settings tab, under Networking, click
Generate Domain
Deploy from the Railway CLI
This step assumes you have the latest version of the Railway CLI installed.
On your local machine -
-
Open your terminal and change directory to the
fastApi
folder -
Link to the Railway project and service by running the following command -
Follow the prompts selecting the correct project name and environment (click here for a reference), and choose the
fastApi
service. -
Deploy the app by running the following command -
Checkpoint
Congrats! You should now have a Mongo replica set and a Python FastAPI app connected to it. Your project in Railway should look something like this -
This is a very basic implementation of the Mongo replica set, so be sure to familiarize yourself with the MongoDB documentation to understand how you might want to tailor it to your needs.
3. Test Mongo Replication
At this point, you should have a Mongo replica set running in Railway as well as a Python application connected to it.
Let's test the connection to Mongo -
- Open your
fastApi
service settings - Click on the domain to open the app in your browser
- The
/
route connects to the Mongo replica set and inserts a single document
- The
- Refresh your browser a few times to add more data to Mongo
- Access the
/get
route to retrieve all of the documents you have just added
Now let's perform some chaos engineering to ensure Mongo replication is working -
- Open the
mongo1
service settings - Open the three-dot menu on the active deployment and click
Remove
to remove the deployment - Access the Python app's
/get
route to confirm it continues working
You can also check out the Mongo logs for detection of node failure and leader elections!
4. Set up Monitoring of the Replica Set
Finally, let's set up some monitoring so you can keep track of the health of your replica set.
Add the Mongo Exporter Service
There are various ways you could retrieve metrics data from the Mongo instances, but we'll use the MongoDB exporter provided by Percona.
- Add a New service by clicking
+ New
in your Railway project - Select
Docker Image
as the Source - Add 'percona/mongodb_exporter:0.40.0' and hit Enter
- In the service Settings, rename the service to
mongo-exporter
- In the service Variables, add the following -
- In the service Settings, add the following Start Command -
- Click
Deploy
This service will connect to the MongoDB and retrieve various metrics. It makes those metrics available on a /metrics
route which Prometheus will use to retrieve them.
Add the Prometheus Service
- Add a New service by clicking
+ New
- Select
Template
as the Source - Type Prometheus, and select the Prometheus template (be sure to select this one)
- Click
Deploy Template
to apply and deploy the service
Now we need to configure Prometheus to scrape metrics from the exporter.
- In the service Settings, eject from the Upstream Repo by clicking the
Eject
button - Choose your Github organization
- Click
Eject Service
- Open the repository that was created for you in Github
- Navigate to the
prometheus.yml
file - Click the pencil icon to edit in place, and add the following job - Note: If you find that the exporter is not retrieving data from Mongo as expected, confirm the internal domain in the Service Settings
- Click
Commit
to commit the changes and redeploy the service in Railway
Add the Grafana Service
- Add a New service by clicking
+ New
- Select
Template
as the Source - Type Grafana, and select the Grafana template
- Configure the required variables to your liking
- Click
Save Config
- Click
Deploy
to apply and deploy the service
Once Grafana is up and running, let's connect it to Prometheus.
- Access Grafana by clicking on the domain from the service settings
- Login with the credentials you configured in the steps above (If you forgot the credentials, you can access them from the service variables tab).
- Click on the hamburger menu in the top left-hand corner
- Expand Connections
- Click Data Sources
- Click Add Data Source
- Find and click Prometheus
- In the Connection section, add the following Prometheus server URL -
- Scroll down to the bottom and click
Save & test
Grafana is now set up to retrieve data from Prometheus! Let's add a dashboard to visualize the MongoDB data.
- Click on the hamburger menu in the top left-hand corner
- Click Dashboards
- Click Create Dashboard
- Click Import Dashboard
- Copy the dashboard.json from the template repo
- Paste the json into the input box and click Load
- Select the prometheus data source and click Import
Nice! You should have a dashboard now that looks something like this -
Conclusion
Congratulations! You have deployed a Mongo replica set, connected a Python FastAPI, and set up monitoring of the replica set using Prometheus and Grafana.
Hopefully you have learned something new or at least gotten a start on implementing your own application stack with Mongo as your highly available, redundant database.
Note that is a very basic implementation, and you should refer to the MongoDB, Prometheus, and Grafana documentation for information on how to customize your implementation.
Edit this file on GitHub