How to Copy a Project from Local Computer to a Remote Server (Using SCP & rsync)

Introduction

To run your AI/ML project on an HPC server, you first need to transfer your files from your local computer to the remote server. The two most efficient and commonly used methods are:

• scp (simple and quick)
• rsync (faster and smarter for large projects)

1. Method 1: Using SCP (Simple and Direct)

What is SCP?

scp (Secure Copy) is a command-line tool used to securely transfer files over SSH.

# command
scp -r <local_folder> username@server_ip:/home/username

# example (the ip address and username used are dummy)
scp -r my_project kunal@192.168.1.10:/home/kunal

# What does this command does 
# ===================================
# Copies entire folder my_project
# Transfers it to the server
# Keeps folder structure infact 
# ===================================

# When to use SCP 
# ===================================
# small to medium projects
# one time transfers
# quick uploads 
# ===================================

local machine output

remote server output (the complete folder is copied to the remote server)

2. Method 2: Using rsync (Recommended for Large Projects)

What is rsync?

rsync is an advanced file transfer tool that only copies changed files, making it faster and efficient.

# note rsync is the linux tool you need linux distribution to run 
rsync -avz my_project username@server_ip:/home/username/

# example
rsync -avz my_project kunal@192.168.1.10:/home/kunal/

# What Makes rsync Better
# ===================================
# Faster for large data
# Can resume interrupted transfers
# Avoids re-copying unchanged files

# with progress bar

rsync -avz --progress my_project username@server_ip:/home/username/

output of the local files sending to the server

output of the server files are copied

3. Verifying Transfer

# After copying, login to server:
ssh username@server_ip
cd ~/my_project
ls

4. Best Practices

# Always copy to your home directory:
/home/username/

# Organize your work:
~/my_project

# Avoid system directories:
/etc /root /usr

Conclusion

Transferring your project to the server is the first step in running workloads on an HPC system. While scp is simple and effective, rsync is the better choice for large or repeated transfers.