Introduction
Copying files from one machine to another is pretty straightforward when you have physical access to both machines. If you are using a Cloud Server, and have no physical access to the machine you want to send a copy of your files to, use the SSH protocol, the safest way for this type of transfer .
Secure Copy Protocol (SCP) is a protocol that relies on this type of connection, while ensuring integrity and security and sending files from a source to the recipient. In addition to making this type of copy by means of a simple shell, many programming languages provide modules and libraries to include this operation in their projects.
In this tutorial you will see how to send and receive files and folders in Node.JS, regardless of the operating system of the source and destination machine.
To proceed, the Node.js framework has to be already installed on your system. If you haven't done so yet and are using an Ubuntu Linux server, follow our guide on How to Install Node.js on Ubuntu 18.04 .
Installing Node.js modules
The “node-scp” module, which can be installed using the Node Packet Manager, allows you to transfer the file via SCP to Node.js. Now, open the terminal and proceed with its installation using the command:
npm i node-scp
In case you also need to indicate the private key to connect via SSH to the recipient device, install an additional module to have your ".pem" key sent by the script. In this case, proceed with the installation of the "fs" module by:
npm i fs
Remote server definition
Within your project, start by defining all the parameters for the recipient as if you were connecting via SSH.
const scp = require('node-scp')
//var fs = require('fs');
var remote_server = {
host: 'your host', //remote host ip
port: 22, //port used for scp
username: 'username', //username to authenticate
password: 'password', //password to authenticate
// forceIPv4: boolean, //Connection allow only via resolved IPv4 address (true/false)
// forceIPv6: boolean, //Connection allow only via resolved IPv6 address (true/false)
// privateKey: fs.readFileSync('./key.pem'),
// passphrase: 'your key passphrase',
}
The variables scp and fs represent in Node.JS an instantiation of the newly installed libraries. Through these variables accessing the methods provided by the libraries, respectively, to send/receive files and to access the files saved on the client will be possible.
The "remote_server" object instead represents the recipient host to be queried and, specifically:
- host: IP address of the receiving host;
- port: number of the port on which to connect (by default, port 22 is used for the ssh/scp connection);
- username: username of the user to login on the recipient host;
- password: password of the user to login on the recipient host;
- forceIPv4 (OPTIONAL): in case you need to define the connection using IPv4 only, set this parameter to "true";
- forceIPv6 (OPTIONAL): in case you need i to define the connection using IPv6 only, set this parameter to "true";
- privateKey (OPTIONAL): in case you need to define the key to connect to the recipient, uncomment this line and use the file path ‘key.pem’ of the source host as argument of the ‘fs.readFileSync’ function ;
- passphrase (OPTIONAL): in case you need to define the passphrase for login to the recipient host, uncomment this line and set the passphrase as a string of this attribute.
Promise vs Async-Await
Before proceeding by showing the different types of sending and receiving files and folders, understanding the difference between the concepts of "Promise" and "Async-Await" is essential.
Sending files with Promise
Using the following code snippet, with the Promise mechanism, you can send the file (whose path is defined in the “local_file_path” variable) to the recipient host by saving it in the path shown in the ‘destination_file_path’ variable.
var local_file_path = './test.txt';
var destination_file_path = '/workspace/test.txt';
send_file_using_promise(local_file_path, destination_file_path);
function send_file_using_promise(file_path, destination_path){
scp(remote_server).then(client => {
client.uploadFile(file_path, destination_path)
.then(response => {
client.close()
})
.catch(error => {})
}).catch(e => console.log(e))
}
The "send_file_using_promise" function will attempt a connection to the previously defined server (remote_server) and, if successful, will upload the file to the preset path.
At the end of the sending, it will close the connection and release the command to Node.JS for the execution of any further functions provided.
Sending files with Async / Await
Similarly to what described above, this portion of code will also upload the file specified in the local_file_path variable to the "remote_server" host in the path indicated in the "detination_file_path" variable.
var local_file_path = './test.txt';
var detination_file_path = '/workspace/test.txt'; send__fileusing_async_await(local_file_path, detination_file_path); async function send_file_using_async_await(file_path, destination_path){
try {
const c = await scp(remote_server)
await c.uploadFile(file_path, destination_path)
c.close()
} catch (e) {
console.log(e)
}
}
In this case, however, the operation, whose logic is still timed through the "await" placed inside the "send_file_using_async_await" function is asynchronous,
Downloading files with Promise
To download the file with Promise, use the following code:
var local_file_path = './test.txt';
var detination_file_path = '/workspace/test.txt';
download_file_using_promise(detination_file_path, local_file_path);
function download_file_using_promise(file_path, destination_path){
scp(remote_server).then(client => {
client.downloadFile(file_path, destination_path)
.then(response => {
client.close()
})
.catch(error => {})
}).catch(e => console.log(e))
}
The SCP protocol is bidirectional: just as just as you can upload a file to a recipient you can download it from a machine to your client. The machines then exchange the roles of source and recipient. This exchange is represented by the replacement of the “client.uploadFile” method with “client.downloadFile”.
The flow logic remains unmodified with the difference that the “text.txt” file is no longer sent from your machine to what was previously the recipient host but the host acquires it on your machine. Note that the parameters passed to the function are inverted with respect to the previous function.
Downloading files with Async / Await
Similarly, the file just shown can be downloaded using async-await:
var local_file_path = './test.txt';
var detination_file_path = '/workspace/test.txt';
download_file_using_async_await(detination_file_path, local_file_path);
async function download_file_using_async_await(file_path, destination_path){
try {
const client = await scp(remote_server)
await client.downloadFile(file_path, destination_path)
client.close()
} catch(e) {
console.log(e)
}
}
The invocation of the “download_file_using_async_await” function will attempt to connect to the client, will wait for the release of the command given by the connection to it and will download the file in the path specified in the second parameter passed to the function. Once downloaded, the file will be saved in the path indicated in the “local_file_path” variable.
Uploading folders
Like files, scp can be used to transfer entire folders from one machine to another.
An implementation for uploading folders is here shown as an example, using both Promise and Async / Await:
Promise
var local_folder_path = './local/dir';
var detination_folder_path = '/server/path';
send_folder_using_promise(local_folder_path, detination_folder_path);
function send_folder_using_promise(folder_path, destination_path){
scp(remote_server).then(client => {
client.uploadDir(folder_path, destination_path)
.then(response => {
client.close()
})
.catch(error => {})
}).catch(e => console.log(e))
}
Async / Await
var local_folder_path = './local/dir';
var detination_folder_path = '/server/path';
send_folder_using_async_await(local_folder_path, detination_folder_path);
async function send_folder_using_async_await(folder_path, destination_path)
{
try {
const client = await scp(remote_server)
await client.uploadDir(folder_path, destination_path)
client.close()
} catch (e) {
console.log(e)
}
}
Verify the existence of paths
Finally, let's see how to find out whether or not a certain path exists on our remote machine.
This feature could be very useful to check the actual uploading of files or folders. It could also be used as a final check of your script, by trying to retry the upload in case of any errors during the transfer.
Promise
var path_to_be_checked = '/server/path';
check_if_path_extists_using_promise(path_to_be_checked);
function check_if_path_extists_using_promise(path_to_check){
scp(remote_server).then(client => {
client.exists(path_to_check)
.then(response => {
client.close()
})
.catch(error => {})
}).catch(e => console.log(e))
}
Async / Await
var path_to_be_checked = '/server/path';
check_if_path_extists_using_async_await(path_to_be_checked);
async function check_if_path_extists_using_async_await(path_to_check){
try {
const client = await scp(remote)
await client.exists(path_to_check)
client.close()
} catch (e) {
console.log(e)
}
}