How to Create a Paginated API with MongoDB and Node JS

Introduction

Have you been wondering how to paginate your server response from your Node Js server when working with a MongoDB database, and you have little or nothing to show for it? Look no further because this article will show you a step-by-step guide to create a paginated response with Node Js and MongoDB and explain the importance of data pagination from the server. Pagination is the process of breaking or dividing an extensive array of data into smaller sizes and returning the requested part of the entire as requested by the client to improve application performance.

Requirements

This article is written so that any newbie may follow along, but some fundamental prerequisites are required.

• Basic familiarity with JavaScript/Node JS is required.
• Install node Js on your machine.
• You must have MongoDB installed on your computer and a basic understanding of MongoDB.

Why Paginate an API Response

Paginating the API response from a server has many benefits; some are outlined below;

• Speed - When compared to fetching just tens of data from a database and returning it to a client, the time required to fetch a large amount of data from a database and return it to a client is longer; pagination can help an application speed up.
• Good UI/UX - Imagine how stressful and awful it would be for a user to see hundreds of thousands of rows of data on a single page; pagination aids in providing a good user experience and user interface.
• Conservation of Client's Memory - A higher percentage of web application visitors access web applications via mobile devices, which have limited memory and computing power when compared to a web server. If pagination is performed on the browser, it can cause our program to act slowly and exhaust the client's resources.
• Eliminates Time Wastage - Most users are only interested in getting the most recent data and not all the data from the database. Paginating the response from a server to send just a few tens saves both the user and computer's time. Implementing Our Pagination

To get started, make a new folder on your computer and call it whatever you like. We'll use the famous express library to build our server and mongoose to communicate with the MongoDB database. Open the folder in your terminal and run the following command.

npm init --y
npm i express mongoose

After installing the express and mongoose packages, we can build a primary express server and connect to our MongoDB database, gradually making the pagination.

const express = require("express");
const mongoose = require("mongoose");
const app = express();

mongoose
  .connect("mongodb://localhost:27017/pgtest")
  .then(() => {
    console.log("Connected to MongoDB Successfully");
  })
  .catch((err) => {
    console.log("Could not connect to MongoDB");
    console.error(err);
  });
const PORT = process.env.PORT || 2020;

app.listen(PORT, () => console.log(`App listenig on port ${PORT}`));

I have created an empty database called pgtest with no data currently stored in it; to have working data, I will download and export the first sixty posts data from JSON placeholder as an array from my local JavaScript file; if you already have some data in your database to work with, you can use the data you have. Jsonplaceholder is a popular free fake rest API for testing and prototyping. You can obtain the data by visiting https://jsonplaceholder.typicode.com.

const mongoose = require("mongoose");
const postSchema = new mongoose.Schema({
  title: {
    type: "string",
    required: true,
  },
  body: {
    type: "string",
    required: true,
  },
  userId: {
    type: "string",
    required: true,
  },
});
const PostModel = mongoose.model("posts", postSchema);
exports.PostModel = PostModel;

We can now proceed to create some posts in our database. I have created a simple post schema and a model called PostModel to generate and read the posts documents.

const express = require("express");
const mongoose = require("mongoose");
const { PostModel } = require("./models/PostModel");
const { PostList } = require("./posts");
const app = express();

mongoose
  .connect("mongodb://localhost:27017/pgtest")
  .then(() => {
    console.log("Connected to MongoDB Successfully");
  })
  .catch((err) => {
    console.log("Could not connect to MongoDB");
    console.error(err);
  });
const PORT = process.env.PORT || 2020;
const db = mongoose.connection;

db.once("open", async () => {
  if ((await PostModel.countDocuments().exec()) > 1) return;
  Promise.all(
    PostList.map((post) => {
      return PostModel.create({
        title: post.title,
        body: post.body,
        userId: post.userId,
      });
    })
  )
    .then(() => {
      console.log("Post added successfully");
    })
    .catch((err) => {
      console.log(err);
    });
});

app.get("/posts", async (req, res) => {
  try {
    const pageNumber = parseInt(req.query.pageNumber) || 0;
    const limit = parseInt(req.query.limit) || 12;
    const result = {};
    const totalPosts = await PostModel.countDocuments().exec();
    let startIndex = pageNumber * limit;
    const endIndex = (pageNumber + 1) * limit;
    result.totalPosts = totalPosts;
    if (startIndex > 0) {
      result.previous = {
        pageNumber: pageNumber - 1,
        limit: limit,
      };
    }
    if (endIndex < (await PostModel.countDocuments().exec())) {
      result.next = {
        pageNumber: pageNumber + 1,
        limit: limit,
      };
    }
    result.data = await PostModel.find()
      .sort("-_id")
      .skip(startIndex)
      .limit(limit)
      .exec();
    result.rowsPerPage = limit;
    return res.json({ msg: "Posts Fetched successfully", data: result });
  } catch (error) {
    console.log(error);
    return res.status(500).json({ msg: "Sorry, something went wrong" });
  }
});
app.listen(PORT, () => console.log(`App listenig on port ${PORT}`));

After the database connection was successful, I listened for the first time a reference was made to the database; if there are no posts in the database, posts are added into the database; lines 17 to 36 of the code are not required if you already have some data in your database to work with.

To paginate the response before sending it to the client, we first convert the pageNumber variable from its request query object to an integer. We also convert the limit variable from the client's request query object to an integer to return to the client the size of data requested by the client. We've also established a default pageNumber and a limit if the client doesn't send it. The last object is dynamically attached to the result object only if the startIndex variable is more significant than zero; this is to enable or disable the client from requesting to get the previous page; similarly, the next object is allowed to make a client request to get the next page of data if the endIndex variable is not greater than the total number of posts in the collection. The skip method allows the query to begin at the correct index, and the limit method, before it is sent to the client, limits the query to the limit supplied by the client.

Conclusion

We now understand pagination, why it is essential to paginate our response and implement pagination in a Node Js server using a MongoDB database. The pagination logic implemented in this article can also be extracted into middleware and used across other Express API routes with similar responses, which is relatively easy to do. You can download the complete source code on GitHub.