1

I am trying to use cloud-storage for GWC tiles. However performance is abysmal, no matter the tweaks or provider I'm using.

What I've tried:

  • Providers: AWS, Wasabi
  • Access: S3 Plugin, File-system plugin with mounted drive using goofy, File-system plugin with mounted drive using s3fs
  • Preseeding tiles: using either of the above methods was too slow (max 128 tiles/sec). I found some success using the local drive to pre-seed, and then using rclone to transfer to cloud with increased connections.
  • Retrieving tiles: after pre-seeding, retrieval is between 300-600ms, even while using memory caching. Compared to local disk access, which is between 150-300ms.

If anyone has had success using cloud storage for GWC tiles, what was the trick that enabled it?

Improve this question
6
  • why would expect the cloud to be faster than a local disk? seeding speed is mostly limited by the cpu on the seeding machine not disk speed
    – Ian Turton ♦
    May 16, 2022 at 14:01
  • @IanTurton, I don't expect it to be faster, I expect it to be usable and on-par, otherwise I think nobody would bother to develop a plug in for something that can't be used. The seeding speed I'm referring to the post compares the same seeding machine (same cpu), when used to seed local tiles and cloud-storage tiles.
    – pkExec
    May 16, 2022 at 14:41
  • you do understand that a community extension is designed as a demo or initial implementation - maybe you need to jump in as owner of the extension and help speed it up
    – Ian Turton ♦
    May 16, 2022 at 14:53
  • Just to make sure, the server is located in the amazon cloud, and in the same region as the S3 bucket, right? Otherwise, slowness would be pretty normal....
    – Andrea Aime
    May 17, 2022 at 16:31
  • @AndreaAime, no the server is not located in the same network. However, since rclone can actually write fast enough using multiple connections, shouldn't the rest of the mechanisms also be up to that speed? My hunch (emphasize on hunch), is that a) the plugin isn't opening up multiple connections and that b) goofy on the default settings have a connection cap which is less than rclone's, thus the less throughput.
    – pkExec
    May 18, 2022 at 7:19

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.