Uploading File to Azure BLOB using Node.js and Express-FileUpload

-

 Uploading File to Azure Blob using Node.js and Express-FileUpload 

Recently, I was working on an assignment for storing of the  information captured from the client application on the Azure cloud. This application is developed using JavaScript Full-Stack with Express and Node.js on the server-side. I was supposed to accept images from client and upload them to Azure Blob Storage using Node.js application.  To implement this, I have explored various ways like Multer package and Express-FileUplaod. After doing some research I have implemented the requirement. But then I have decided to put my thoughts in the form of this article.  In fact, I have already written an article on similar requirements which you can read it from this link. 

In this article, we will see how to use an express-fileupload to upload file to Node.js Server Application  and further uploading it in Azure Blob. 

The figure 1, will explain an idea of the application implementation





Figure 1: The idea of file upload application implementation 

The numbering used in the image explains all the steps for implementation of the application. 

Express-FileUpload

The express-fileupload is a simple middleware that is used to upload files to the server when using Node.js Express applications. This middleware is added into the express HTTP request processing object and it reads the posted  file from the client using request object and processes it. 

Azure Blob Storage

The uploaded file will be further stored in the Azure Blob Storage. The Azure Blob storage is a Microsoft's Object Storage solution for the Cloud. The Azure Blob storage is capable for storing massive amount of unstructured data like images, documents, audio, video files, etc.  To access the Azure Blob storage, we need an Azure Subscription. The Azure subscription can be created using Azure Free account by using this link.  

Creating a Storage Account  to Access Blob Storage

Step 1: To create a storage account, we need to create Resource Group. Open the Azure Portal and login on it.  To create a resource group, click on the Resource Group from the home page as shown in Figure 2

The Resource Group is an logical group of all the resources (App, Database, storage) you create on Azure.

Figure 2: Creating a resource group  

Once the Resource Group link is clicked, the Resource Group page will be displayed. On this page, click on Create link. This action will show Create Resource Group page. On this page enter the resource group information like Subscription, Resource Group Name, Region, etc. and click on Review + create  button as shown in figure 3.



Figure 3: Creating a Resource Group  

Step 2: . Select the Storage accounts from the Home page as shown in figure 4



Figure 4: The Storage Account      

Once the Storage accounts is clicked, the page for creating new storage account is displayed as shown in figure 5



Figure 5: Creating Storage account          

Click on the Create link and this will show the Create a storage account page.  On this page enter details like Subscription, Resource Group  (we have created it in Step 1), Storage account name, Region etc. and then click on the Review + create button. The Storage account will be created.  The Storage account name we are creating is msitart. The details are shown in figure 6



Figure 6: Create storage account   

Once, the storage account is created, click on it, it will show all storage properties, e.g. Containers, Queues, Tables, File Shares, Access Keys, etc. as shown in the figure 7



Figure 7: Storage account properties 

On this page, click on Access Keys, these keys are used to authenticate the client application to access Storage Account. Click on Show Keys link, this will show Connection string as shown in figure 8



Figure 8: The Connection Strings  

Copy the Connection String and paste it in the notepad, we will be using this in the Node.js application.

As shown in Figure 7, click on Containers link and create anew container of name imagecontainer. (Make sure that the container name is in lower case.)

Creating the Node.js application

Step 3: Create a new folder of name expressfileupload. Open this folder in Visual Studio Code. Open the Command Window (or Terminal window in Linux and Mac) and navigate to the expressfileupload folder. We need to create a package.json file and install various package which we will be using for the application development.

Run the following command to create package.json file in the folder

npm init -y

Run the following command to install required packages

npm install --save express express-fileupload into-stream dotenv azure-storage

These packages are used for the following requirements

  1. dotenv, since we will be storing the Azure Storage Account Connection String in Node.js environment file we need this package to configure the Node.js application to read keys from the environment file.
  2. into-stream, This package is used to convert promise, string, array, buffer, arraybuffer, object, etc. into a stream.
  3. azure-storage,  This is a Microsoft's Storage SDK for Node.js to access Azure Storage in Node.js applications.
Step 4: Modify the package.json to enable ES 6 Module syntax to import JS modules in the application code as shown in listing 1:

{
  "name": "expressfileupload",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "start": "nodemon server.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "azure-storage": "^2.10.5",
    "dotenv": "^10.0.0",
    "express": "^4.17.1",
    "express-fileupload": "^1.2.1",
    "into-stream": "^7.0.0"
  },
  "type": "module"
}

 
Listing 1: Modifications in package.json to support ES 6 syntax for Module import

Step 5:  In the project add a folder and name it as files. This folder is used by the express-fileupload to  save the posted file from the client application.

Step 6: In the project add a new file and name it as '.env'. This is an environment file where we will define Name=Value pairs of the environment variables used by the Node.js application. It is recommended that, instead of hard-coding security critical values in the application code we should put them in environment file.

AZURE_STORAGE_CONNECTION_STRING= "YOUR CONNECTION STRING HERE"

Listing 2: An Environment file

Step 7: In the project add a new file and name it as server-blob.js. In this file add the code as shown in the listing 3

import express from "express";
import fileUpload from "express-fileupload";
import path from "path";
import { fileURLToPath } from "url";
import azureStorage from "azure-storage";
import intoStream from "into-stream";
import dotenv from "dotenv";

// 1.
const __filename = fileURLToPath(import.meta.url);
// 2.
const port = process.env.PORT || 7001;
// 3.
const instance = new express();

// 4.
const containerName = "imagecontainer";

// 5.
const __dirname = path.dirname(__filename);
// 6.
dotenv.config();
// 7.
instance.use(
  fileUpload({
    createParentPath: true,
  })
);
// 8.
const blobService = azureStorage.createBlobService(
  process.env.AZURE_STORAGE_CONNECTION_STRING
);
// 9.
instance.get("/", (req, res) => {
  res.sendFile(path.join(__dirname, "index.html"));
});
// 10.
instance.post("/fileupload", (request, response) => {
  // 10.1.
  if (!request.files) {
    return res.status(400).send("No files are received.");
  }

  // 10.2.
  const file = request.files.file;
  // 10.3.
  const path = __dirname + "/files/" + file.name;
  // 10.4.
  file.mv(path, (err) => {
    // 10.5.
    if (err) {
      return response.status(500).send(err);
    }
    // 10.6.
    return response.send({ status: "success", path: path });
  });
});
// 11.
instance.post("/blobupload", (request, response) => {
  if (!request.files) {
    return res.status(400).send("No files are received.");
  }

  // 11.1.
  const blobName = request.files.file.name;
  console.log(`Blob Name ${blobName}`);
  // 11.2.
  const stream = intoStream(request.files.file.data);
  console.log(`stream ${stream}`);
  // 11.3.
  const streamLength = request.files.file.data.length;
  console.log(`Length ${streamLength}`);
  // 11.4.
  blobService.createBlockBlobFromStream(
    containerName,
    blobName,
    stream,
    streamLength,
    (err) => {
      if (err) {
        response.status(500);
        response.send({ message: "Error Occured" });
        return;
      }

      response.status(200).send({message:
'File Uploaded Successfully'});
    }
  );
});
// 12.
instance.listen(port, () => {
  console.log(`Server is listening on port ${port}`);
});

Listing 3: The Application code

The code in listing 3 shoes that it uses ES 6 module import to import modules required for the application e,g, express, express-fileupload, into-stream, azure-storage,dotenv, etc. and further the code  has specifications as explained in following point. (Please note that following numbering matches with the comment applied on the application code)

  1. The support for __dirname is not available in Node.js with ES 6 support, so use the fileURLToPath() function to read the directory path.
  2. The port from which REST API will be accessible
  3. Defining instance of express
  4. Defining the container name where uploaded files will be stored
  5. The directory path, this will be the path for the  directory where the server is running
  6. Environment configuration to read keys from the .env file 
  7. configure the file upload middleware
  8. Connecting the BLOB Service using the Connection String read from the Environment Key from .env file
  9. The get request to respond index.html. This file contains HTML UI that will be posting the image file to server. 
  10. The post request to to accept posted file from the client  
    1. If the file not found then respond error
    2. Read the file
    3. The file path where the file will be stored on the server
    4. Copy the received file in 'files' folder
    5. If error occurred then send error response
    6. Send the success response
  11. Post request for accepting file from the client and uploading it on Blob storage 
    1. Read the file name received from the client 
    2. Convert the file into stream
    3. Read the Length of the file
    4. Upload the file from to the Blob
  12. Start listening the server on the port

Step 8: In the project add a new file and name it as index.html, add the code in this file as shown in Listing 4


<!DOCTYPE html>
<html>
  <head>
    <title>Uploading The File Using Express</title>
  </head>
  <body>
    <h1>Uploading The File Using Express</h1>
    <form method="POST" action="/blobupload" enctype="multipart/form-data">
      <input type="file" name="file" />
      <input type="submit" />
    </form>
  </body>
</html>

Listing 4: index.html

The index.html contains a HTML form with HTML input file element and a submit button. The file that is to be uploaded is selected using the input file element that has name attribute as file. we have used this attribute value in the server code to read uploaded file (see listing 3, code comment 11.1). When the form is submitted, the REST API with endpoint as blobupload will be requested with file posted to it.  

To run the application, run the following command from the command prompt

node server-blob.js

This will start the server on the port 7001. Open the browser and browse the address http://localhost:7001. The index.html will be loaded in the browser as shown in the figure 9



Figure 9: The index.html 

Click on the Choose File button, this will open the Open file dialog, from this dialog select the file to be uploaded, and then click in Submit button. The file will be uploaded, if the file upload is successful, the File Uploaded Successfully message will be displayed.


Visit back to the Azure portal and open the Containers as shown in figure 7. If you open the imagecontainer, you will see the file uploaded to Azure Bob Storage.

The Code for this article can be downloaded from this link.

Conclusion: The Express-FileUpload package is a great use for uploading files in Node.js application. Using Azure-Storage package we can easily upload files on Azure Blob Containers. This helps to build JavaScript Full-Stack apps for cloud easily.     


 

Popular posts from this blog

Uploading Excel File to ASP.NET Core 6 application to save data from Excel to SQL Server Database

-
Image
Recently, while conducting a session on ASP.NET Core 6, my client raised a query regarding uploading excel files to the ASP.NET Core MVC application and then further saving data from excel to the SQL Server Database table. I gave him the solution and then thought to write a post on it. As we all know, the Excel Workbook is one of the most heavily used files to save data. But to persist data it is recommended that it should be stored in an SQL Server database.  If the data is stored in an SQL Server database it can be queried easily.  Figure 1, explains an implementation of the application which we are going to discuss in the article. Figure 1: The ASP.NET Core That accepts Excel files and then further store data from Excel Files in SQL Server Database The   ExcelDataReader.DataSet Library To read excel files in ASP.NET Core applications, we need to use the ExcelDataReader.DataSet package. This is a lightweight and fast library for reading Excel files in .NET applications. This is writt
Read more

ASP.NET Core 6: Downloading Files from the Server

-
Image
In this article, we will implement the file downloading feature using ASP.NET Core 6 application. In real-world applications, we can have various files like reports, spreadsheets, images, PDF documents stored on the server. (Note that all cloud-hosted applications prefer to store these files on the storage services, e.g. Blob, S3, etc.). But what if the client prefers to store files on the separate file server mapped to the web application host server? Then, in that case, we may need to download these files from the file server. In this article, we will implement exactly the same thing.   Step 1: Open Visual Studio 2022 and create a new ASP.NET Core MVC Application targetted to .NET 6. Name this application as Core6_FileDownload. In this project add a new folder and name it as ServerFiles. In this file add some images. (You can add Excel, PDF, Word, Files). Step 2: Since we need to read the ServerFiles folder using the ASP.NET Core application, we need to add the IFileProvider service
Read more

ASP.NET Core 6: Using Entity Framework Core with Oracle Database with Code-First Approach

-
Image
In this article, we will see how to use Oracle Database with Entity Framework Core with Code-First Approach. The reason behind writing a post on this is, recently while conducting training, one of my students raised the query regarding the EF Core with Oracle. Since Oracle is one of the most powerful databases, various enterprise applications are using it for the past several years and since the .NET Core is the most popular technology for building modern apps, the scenarios for accessing the Oracle database in .NET Core apps the use Entity Framework Core is common. Oracle Data Provider for .NET Entity Framework Core The Oracle Data Provider for .NET i.e. ODP.NET is used to access data from the Oracle database in the .NET Core apps using Entity Framework Core. This uses the Oracle.EntityFrameworkCore.dll, a managed library. This data provider allows using Code-First and Database-First approaches to perform Read/Write Operations with Oracle database.    Implementation To implement code
Read more