Make any Iterable into a readable stream

NodeJS streams are one of the most powerful / often misunderstood / least understood concept ever. People often use streams without understanding how it works.

Any NodeJs beginners might have come across the below code during their learning phase.

[embed]

It teaches beginners the basics of using streams. It’s self-explanatory and sadly it just ends there.

Ever wondered about implementing a readable stream from any data source? In this article we are going to take a step forward into understanding an elegant way of creating custom readable stream. For the sake of simplicity, lets always pipe to stdout.

Problem statement

We are going to build a “lorem ipsum” app, which streams random “lorem ipsum words” into stdout. A simple straight forward solution could be below

[embed]

It works but there is a much more beautiful way to do the same. lets split up the solution into below steps

1. ES6 Generator to rescue

lets first write an “ES6 generator” that yields random “lorem ipsum” word

[embed]

2. Make the Generator Async

To make things more interesting, let’s make “async loremIpsumGenerator” that “yields” the word after a random delay.

For await … wait what? Note the usage of “for await” in combination with async iterable returned from the async generator.

[embed]

Putting it all together

NodeJS v12 introduced a new feature that enables us to create a readable stream from any async (or) sync iterable. This feature helps us write clean, elegant code to implement readable streams.

[embed]

One might wonder the reason behind random wait. I added it to simulate readable streams from realtime data sources. An example would be to stream data from IoT devices, where there would be a continuous stream of data with a variable data flow rate.