TransWikia.com

Faster Data Transfer?

Data Science Asked by Andrew Maurer on May 3, 2021

I’m doing a project which involves a small dataset (4GB). I’m trying to upload it to Paperspace to do some analysis but it’s taking an absurd amount of time.

  • Using Gradient, it would upload maybe 40MB in 10min then stop uploading. This was done using the “upload” button in the jupyter notebook.
  • Using a P400 hourly computer I’m transferring at 800MBPS over SSH (well, through Nautilus), but it’s still taking hours to transfer this small dataset.

What am I doing wrong? Is there a faster way?

One Answer

Most cloud systems support the command line mv command to move files or folders.

Taking a command line approach allows for more control and visibility into tasks.

Answered by Brian Spiering on May 3, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP