Tutorial – Create a Block Explorer for the Libra Cryptocurrency: Part I – The Backend

A block explorer is one of the most important tools every cryptocurrency needs. It allows the end-user to quickly search the blockchain ledger for addresses and transactions data.

During the first month of Libra testnet existence, several experimental block explorers were created by the Libra open-source developers community.

At Libra Startup, we have developed and released our own Libra block explorer. It is open-source, and currently, you are reading its step-by-step tutorial.

This quick tutorial explains how to create a simple Libra block explorer’s back-end with Node.js and MongoDB, communicating with the Libra blockchain through a lightweight JavaScript gRPC library, and returning data to the front-end through a custom made API.

The front-end guide will be available in the second part of this tutorial. Currently, you can use its beta source code, see GitHub.

The block explorer application, created according to this tutorial, is in its early beta stage, live at Libra Checker.

Part I – The Backend

Below are the steps that need to be taken to set up the back-end from scratch.

1. Install Node.js, nginx, configure the firewall
2. Configure a reverse proxy (and SSL)
3. Install MongoDB
4. Create the API script and run it with PM2
5. Install Libra gRPC client to save Libra blockchain data to MongoDB

We are running this on a fresh Ubuntu 19.04 Linux install in a VM on Google Cloud.

The next few sections of this tutorial are partly based on these tutorials: 1 and 2.

Below are the commands you can run to achieve the same result as in the above tutorials. Do not use the root account, create a separate Linux user.

1. Install Node.js, nginx, configure the firewall

Install Node.js:

 curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
 sudo apt-get install -y nodejs
 sudo apt install build-essential

Install and set up nginx:

sudo apt update
sudo apt install nginx

Enable the firewall and start nginx:

sudo ufw enable
sudo ufw allow 'Nginx HTTP'
sudo ufw allow 'ssh'
sudo ufw status
sudo systemctl enable nginx

You can now access your Nginx web server in your browser when visiting the server’s external IP address. The default nginx home page should look similar to the below.

2. Configure a reverse proxy (and SSL)

Configure your hostname, if you want to use it instead of the IP. For Google Cloud, follow this video tutorial.

Now, let’s configure a basic http reverse proxy. Everywhere, change librachecker.com with your domain name or IP address.

sudo nano /etc/nginx/sites-available/librachecker.com

Paste the following code to set up the reverse proxy that will listen to the localhost on 3000 port:

server {
  listen 80;
  listen [::]:80;

  server_name api.librachecker.com;
  location / {
    proxy_set_header   X-Forwarded-For $remote_addr;
    proxy_set_header   Host $http_host;
    proxy_pass         "http://127.0.0.1:3000";
  }
}

Add it to enabled sites:

sudo ln -s /etc/nginx/sites-available/librachecker.com /etc/nginx/sites-enabled/

Now edit nginx.conf to uncomment the following line: server_names_hash_bucket_size 64;

sudo nano /etc/nginx/nginx.conf
sudo nginx -t
sudo systemctl restart nginx

You should now be able to see your nginx’s welcome screen when accessing a domain name with an http protocol, e.g. http://api.librachecker.com.

SSL certificate (optional)
If you are hosting your back-end and front-end on separate servers and want to use a secure https connection, then you also need to install a certificate.

Let’s install the Let’s Encrypt certificate:

sudo add-apt-repository ppa:certbot/certbot
sudo apt install python-certbot-nginx
sudo certbot --nginx -d api.librachecker.com

sudo nginx -t
sudo systemctl restart nginx

Open SSL ports on the firewall:

sudo ufw allow 'Nginx Full'
sudo ufw delete allow 'Nginx HTTP'
sudo ufw status

By now, you should be able to access the same nginx’s welcome screen with https, e.g. https://api.librachecker.com.

More details about the nginx and SSL setup can be found in these two tutorials: 1 | 2

3. Install MongoDB

The idea of the blockchain explorer is that it first saves all transactions from the blockchain to a local database. It then uses this data to display the results for the end-users.

For the end-user to directly query a blockchain would be highly inefficient. That is why a local database is a better idea.

For our block explorer, when fetching the transactions data from the blockchain, the script also creates a separate collection for addresses, where every address has an array of sent and received transactions.

Using this data, the front-end application then builds the addresses and transactions pages on-demand, when requested by the block explorer’s users.

To install and launch MongoDB, run the following commands:

sudo apt update
sudo apt install -y mongodb
sudo service mongodb start
sudo npm link mongodb

The default configuration works great. MongoDB is now installed and ready to be used.

4. Create the API script and run it with PM2

We now need to create the API endpoints:

1. Get a transaction by its ID.
2. Get address balance and recent transactions.
3. Get 50 most recent transactions.

Create a new api.js file with the following code:

TODO: Add explanatory comments for the JS code.

cd ~
sudo nano api.js 
const http = require('http');

const hostname = 'localhost';
const port = 3000;

async function run() {

  const server = http.createServer((req, res) => {

    res.statusCode = 200;

    var MongoClient = require('mongodb').MongoClient;
    var url = "mongodb://localhost:27017/";

    if (req.method === 'POST') {

      let body = '';

      req.on('data', chunk => {
        body += chunk.toString(); // convert Buffer to string
      });

      req.on('end', () => {

        res.setHeader("Access-Control-Allow-Origin", "*");
        res.setHeader('Content-Type', 'application/json');

        if (body.length == 64) {
          console.log('address req');

	  if (body == '0000000000000000000000000000000000000000000000000000000000000000') {
           res.end(JSON.stringify({balance: 'Minting account', txs: []}));
           return;
          }

           MongoClient.connect(url, function(err, db) {
            if (err) throw err;
            var dbo = db.db("libra-local");

            dbo.collection("addresses").findOne({_id: body}, function(err, result) {
              if (err) throw err;

              let balance = 0;
              let txs = [];

              let itemsProcessed = 0;

              if (typeof(result.received) != 'undefined') {

                result.received.forEach(tx => {

                  dbo.collection("transactions").findOne({_id: Number(tx)}, function(err, result2) {
                    if (err) throw err;

                    // add to balance
                    balance += result2.value;

                    txs.push(result2);

                    itemsProcessed++;

                    if (itemsProcessed == result.received.length) {
                      processSent();
                    }

                  });

               });

             }

             else { processSent() }

             function processSent() {

               let itemsProcessed = 0;

               if(typeof(result.sent) != 'undefined') {
                 result.sent.forEach(tx => {

                   dbo.collection("transactions").findOne({_id: Number(tx)}, function(err, result2) {
                     if (err) throw err;

                     balance -= result2.value;

                     txs.push(result2);

                     itemsProcessed++;

                     if (itemsProcessed == result.sent.length) {
                       prepareData();
                     }

                   });

                 });
               }
               else { prepareData(); }
             }

             function prepareData() {
               // console.log(balance, txs)

               txs.sort(function (a, b) {
                 var aNum = parseInt(a._id);
                 var bNum = parseInt(b._id);
                 return bNum - aNum;
               });

               txs = txs.slice(0,50);

               console.log(txs);

               returnData();
             }

             function returnData() {
               db.close();
               res.end(JSON.stringify({balance: balance, txs: txs}));
             }

           });

         });

        }

        else if (body.length < 32) {
          console.log('tx_req');

          MongoClient.connect(url, function(err, db) {
            if (err) throw err;
            var dbo = db.db("libra-local");

            dbo.collection("transactions").findOne({_id: Number(body)}, function(err, result) {
              if (err) throw err;
              db.close();
              result = JSON.stringify(result);

              res.end(result);
            });
          });
        }

      });
    }

    else if (req.method === 'GET') {
      console.log('recent_txs request');

      MongoClient.connect(url, function(err, db) {
        if (err) throw err;
        var dbo = db.db("libra-local");

        dbo.collection("transactions").find({}).sort({_id:-1}).limit(50).toArray(function(err, result) {
          if (err) throw err;
          db.close();
          result = JSON.stringify(result);

          res.setHeader("Access-Control-Allow-Origin", "*");
          res.setHeader('Content-Type', 'application/json');
          res.end(result);
        });
      });
     }

   });

   server.listen(port, hostname, () => {
   console.log(`Server running at http://${hostname}:${port}/`);

  })
}

(async () => {
  console.log(await run());
})()

Install and set up PM2 to run our script in the background and on a startup:

sudo npm install [email protected] -g
pm2 start api.js
pm2 startup systemd

Then check the output in your shell, and run the displayed command to finish setting up the startup script.

5. Install Libra gRPC client to save Libra blockchain data to MongoDB

Libra Blockchain provides Admission Control (AC) public API endpoint to send transactions and query the blockchain from the gRPC clients. This allows executing Libra Core (Rust) CLI commands remotely.

For the Libra block explorer, we will use the fork of the bonustrack/libra-grpc lightweight JavaScript Libra gRPC client. The forked version is prepared for this project’s needs.

cd ~
sudo git clone https://github.com/giekaton/libra-grpc.git
cd libra-grpc
sudo npm install

If you encounter an error, try rebuilding the libra-grpc package. Or run the above commands from the root user.

sudo npm rebuild

TODO: Fix so that root is not necessary.

Now, run the script to save transactions data from the blockchain to your local database. This will take several hours, depending on the transactions count.

TODO: Optimize saveAll.js for speed.

node test/saveAll.js

It should populate the database with the transactions collection from the Libra blockchain, together creating and populating the addresses collection.

You can stop this script at any time with CTRL+C, and then run it again. It will continue fetching transactions from where it stopped the last time.

You need to keep this script running in the background so that after it saves all existing transactions, it will continue to check for new transactions every 0,5 seconds and keep updating the database. To run it in the background, instead of using the ‘node’ command, start it with PM2:

pm2 start test/saveAll.js
sudo pm2 status

Your pm2 status response should look similar to this:

This is how the transactions collection is structured:

And this is the addresses collection:

The above MongoDB UI is the Robo 3T, a shell-centric MongoDB management tool.

After successfully saving the transactions and addresses data, your local MongoDB should look similar to the above. It now has all the data that our simple Libra block explorer needs.

All systems running. You should now be able to access your API endpoints.

The first part of the tutorial is complete!

The front-end application is built using Vue.js. The front-end tutorial (Part 2) will be available soon.

The beta version source code for the front-end is available on GitHub. This build is currently live on https://librachecker.com.

This tutorial is a work in progress. We will continue updating the tutorial, also improving the Libra Checker.