Tired of running out of cloud storage?

“Your cloud storage is at 98% capacity” or “Your iCloud is full” They’re the biggest companies in the world and we still have to pay ?

Why not make a free, open-source cloud storage ?

So I spent the last couple of days working on **hive-cloud **(github).

Photo by Damien TUPINIER on Unsplash

Btw: This project is an insult to all of software engineering’s fundamental rules, and I liked it 😈

It allows you to 👇

1. Store as much data as the “hive” can hold
2. Share download links
3. Encrypt files at upload, so not everyone can access your data.
4. Decrypt files at download ( ~logic~ 🤓 )

Try it here : hive-cloud

Here’s how it works 🐝

You give some of your RAM to the Hive. You have to share a bit of your memory in order to use the Hive. Then, your browser will behave as a storage, it will receive small chunks of everyone’s data, and make sure each chunk is stored on at least 10 different clients (also called “bees” 🐝). This way, if a client disconnects, the other ones will ask each other to store the under-replicated chunks.

During your session, you persist a bit of everyone’s data for a short time, and you broadcast it to other clients.

When you download a file, you ask all clients storing chunks of your file to send them.

That’s the whole point of this project :

Chunks should flow endlessly between clients.

⚠️ One note: It’s really, heavily, using your internet connection.

What could possibly go wrong ?

Unfortunately, lots of things.

That’s why this project is open-source. I won’t be able to push the limit alone and I need people interested in optimization. I know there’s some pretty hardcore devs out there, so I’m hoping one of them sees this article.

1. RAM available < Storage available.

It’s easy to tell that we have less RAM available on earth than persistent storage (HDD, SSD and so on). We also try to replicate each chunk 10 times, so after a few hours of deep calculations, I came to realize that it divides the useful storage capacity by 10.

The fact is, we all have an insane amount of power in our hands. We all have at least 4GB of RAM near us, ready to be used, at all times. I bet we can use 30% of everyone’s RAM and they won’t even notice.

Imagine a PC, originally built for video games, used only to watch YouTube videos, we could use its RAM !

2. Average internet speed

The current average internet speed is around ~109 Mb/s. So to download a 1 GiB file, you’ll have to wait around 80 seconds.

3. Hostile clients (wasps)

And That’s where I broke all software engineering’s fundamental rules: the client is almighty.

When we download a file, we ask multiple clients if they agree on a chunk’s content. If most of them are agreeing on the fact that they have the same chunk, we forward it to the client wanting to download it. If the majority of clients are hostile, we won’t be able to download a file.

Each chunk has an id. This id is the entire file's hash + chunk index so you will never be able to download a corrupted file.

4. But… What if all 🐝 leaves ?

Well, **We’re F***.

To work, the Hive needs a stable or always growing capacity. Otherwise we lose data. There’s no other choices with its current behaviour. It might change, I’m thinking of allowing the server to store a few under-replicated chunks.

This project is experimental

As we saw, a lot of things can go wrong. But for now, it’s working-ish.

I can’t tell you how much I hope you understand these rules :

1. Don’t upload sensitive data.
2. Don’t upload illegal data.
3. Don’t upload data you can’t afford to lose.

Once again, you can become a 🐝 here : hive-cloud

Thank you very much for reading this article, don’t hesitate to give any kind of feedback (here or on github).