node.js - NodeJS: Rapidly converting image with Data URI causes caching, memory leak -

i working on software reads high amount of images , processing on images reading height, width etc. came see difficult situation memory used during converting image url data uri.

request.get(imageurl, function (error, my_data, body) {     var imagebase64data = new buffer(body).tostring('base64');     var imagedata = "data:" + my_data.headers["content-type"] + ";base64," + imagebase64data;     // using imagedata in image processing using graphicsmagick module }); 

now problem facing when node process starts run , process these images, memory used upto 1.7gb. , system starts giving error of memory.

i checked , learned buffer used convert image url data uri not deletes memory once out of scope.

please suggest or how should go further resolve issue.

not sure how translates using buffer base64 encoding. how translate buffer works. when create new buffer increased heaptotal , heapused. though not translate number of blocks being used. deleting buffer reduce size of heap not in proportion number of blocks used in buffer. suggest after you've consumed it, try , set imagebase64data = null. node's internal gc free blocks in next sweep.

also, since variables declared inside request callback handler anonymous function every time make request it's creating new imagebase64data, imagedata in memory. if declare outside request , reference inside new data may overwrite value. need careful mutating values inside function. if want maintain referenced processed url's suggest looking memonizing or caching don't process same url again , again.

here's simple test shows how gc works when set buffer null.

→ node                     > process.memoryusage() { rss: 21639168, heaptotal: 10522624, heapused: 5058760 } > mybuf = require('buffer').buffer { [function: buffer]   poolsize: 8192,   from: [function],   alloc: [function],   allocunsafe: [function],   allocunsafeslow: [function],   isbuffer: [function: isbuffer],   compare: [function: compare],   isencoding: [function],   concat: [function],   bytelength: [function: bytelength] } > var buf = new mybuf(56789) undefined > process.memoryusage() { rss: 24563712, heaptotal: 11571200, heapused: 6459632 } > buf = null null > process.memoryusage() { rss: 24756224, heaptotal: 11571200, heapused: 6591688 } > process.memoryusage() { rss: 24805376, heaptotal: 11571200, heapused: 6637832 } > process.memoryusage() { rss: 23359488, heaptotal: 8425472, heapused: 5567208 } > 

note: nodejs 6.10.2 states creating buffer using

new buffer(body) deprecated.