TransWikia.com

Weird R problem: Internet keeps disconnecting whenever I run any raster calculation using the "raster" package

Stack Overflow Asked by ChongSChoi on November 15, 2021

I ran into what seems to be a fairly unique problem on R’s "raster" package.

I am not sure how to provide a reproducible example for people reading this, but the long and short of it is that whenever I perform calculations involving my raster objects, the internet shuts down for the whole house. While I can’t provide the raster images themselves because they are so big, I can provide their general description. Objects like these are about a gigabyte each.

class      : RasterLayer 
dimensions : 15336, 19016, 291629376  (nrow, ncol, ncell)
resolution : 30.85642, 30.85642  (x, y)
extent     : 610995.9, 1197762, 9526801, 10000015  (xmin, xmax, ymin, ymax)
crs        : +proj=utm +zone=52 +south +datum=WGS84 +units=m +no_defs  +ellps=WGS84 +towgs84=0,0,0
source     : C:/Users/tug74077/AppData/Local/Temp/RtmpcVf7pt/raster/r_tmp_2020-07-21_082952_22832_46671.grd
names      : layer 
values     : 6378.035, 2016403  (min, max)

And sometimes I use raster stacks, which are much larger (about 10 gigabytes)

class      : RasterStack 
dimensions : 15336, 19016, 291629376, 30  (nrow, ncol, ncell, nlayers)
resolution : 30.85642, 30.85642  (x, y)
extent     : 610995.9, 1197762, 9526801, 10000015  (xmin, xmax, ymin, ymax)
crs        : +proj=utm +zone=52 +south +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0 
names      : ap_hp_stack.1, ap_hp_stack.2, ap_hp_stack.3, ap_hp_stack.4, ap_hp_stack.5, ap_hp_stack.6, ap_hp_stack.7, ap_hp_stack.8, ap_hp_stack.9, ap_hp_stack.10, ap_hp_stack.11, ap_hp_stack.12, ap_hp_stack.13, ap_hp_stack.14, ap_hp_stack.15, ... 
min values :     426.50653,     403.31589,     381.38617,    360.64886,     341.03912,     322.49561,     304.96039,     288.37863,     272.69846,      257.87088,      243.84953,      230.59058,      218.05255,      206.19627,      194.98465, ... 
max values :     134839.22,     127507.53,     120574.50,    114018.44,     107818.85,     101956.36,      96412.63,      91170.34,      86213.09,       81525.38,       77092.55,       72900.77,       68936.89,       65188.55,       61644.02, ...

The type of calculation that would disconnect the internet ranges from a simple stacking operation of 30 rasters like this:

ann.cost_hp_t_stack <- stack(ann.cost_hp_t)

(ann.cost_hp_t is a list of 30 rasters that look like a single raster layer in the above (former) description that will be stacked to create ann.cost_hp_t_stack, which resembles a raster stack in the above (latter) description),

to an operation that looks like this:

for (i in c(1:30)){
  ann.cost_hp_t[[i]] <- ann.cost_t_im / ((1 + 0.05)^i)
}

where ann.cost_t_im is another raster layer resembling the raster layer described above.

In addition to the internet cutting out for the whole router/house, my local disk gets filled up too, and I have to regularly restart R to free up about 140 gigabytes of disk space.

If you have read this far, thank you very much for your time. Also, sorry if the formatting is confusing.

TL;DR: My internet keeps cutting out when I use the "raster" package in R to create gigabytes upon gigabytes of data.

One Answer

The raster package does not use your network for normal computations (it only uses it to download data when you use the getData function). So it is directly related to that. It has to be related to what you computer does when files are created.

All I can think of is that you have a system that automatically copies your data to the cloud. So if you create a bunch of big files, that would slow down the Internet.

As for the filling up of your disk; that is because you are using very large files with functions that then need save these to disk (in a temp file). You can use other functions such as calc to use filenames. There would still be files, of course, but you may cut out intermediate files; if there are any. See raster::removeTmpFiles for removing them without exiting R.

Answered by Robert Hijmans on November 15, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP