write and read streams everything you need to know?

node js Streams are collections of data — just like arrays or strings. The difference is that streams might not be available all at once, and they don’t have to fit in memory. This makes streams really powerful when working with large amounts of data, or data that’s coming from an external source one chunk at a time.

1
2
3
4
5
6
7
8


const fs = require('fs');
const file = fs.createWriteStream('./file.txt');
//1e6 = 1000000
for(let i=0; i<= 1e6; i++) {
  file.write('We can Change\n');
}

file.end();

fs.createReadStream() allows you to open up a readable stream in a very simple manner. All you have to do is pass the path of the file to start streaming in. It turns out that the response (as well as the request) objects are streams.

1
2
3
4
5
6
7


const fs = require('fs');

fs.readFile('./file.txt', (err, data) => {
    if (err) throw err;
    console.log(data.toString());
});

The fs.readFile() function buffers the entire file. To minimize memory costs, when possible prefer streaming via fs.createReadStream().

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


const fs = require('fs');
const csv = require('fast-csv');

fs.createReadStream('./file.txt')
    .pipe(csv.parse({ headers: false }))
    .on('error', error => console.error(error))
    .on('data', (data) => {
        console.log(data)
    });

this will read the file.txt

 ['We want change.' ]
[ 'We want change.' ]
[ 'We want change.' ]
[ 'We want change.' ]
[ 'We want change.' ]
[ 'We want change.' ]
[ 'We want change.' ]
[ 'We want change.' ]
[ 'We want change.' ]
[ 'We want change.' ]
[ 'We want change.' ]
[ 'We want change.' ]
 for 1 million time

write and read streams everything you need to know?

See Also