Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Infrastructure Migration
07-20-2017, 05:30 AM,
#1
Posts: 1,163
Threads: 116
Joined: Feb 2011
Reputation: 45
Thumbs Up  Infrastructure Migration
skeight Offline Administrator View worlds
********
I am starting the process of migrating the entire site to the new infrastructure which will likely span the next week.  I am aiming to have this complete by the end of the month.

There will likely be issues with rendering as this process is happening but I will do my best to reduce the downtime of rendering and I will keep you updated on the progress in this thread.

First step is a bulk data transfer of which any changes between now and then will be synced before the full switch over so there won't be many updates until next week, just a waiting game over this weekend.
Quote
07-25-2017, 10:09 PM, (This post was last modified: 07-25-2017, 10:09 PM by skeight.)
#2
Posts: 1,163
Threads: 116
Joined: Feb 2011
Reputation: 45
RE: Infrastructure Migration
skeight Offline Administrator View worlds
********
New servers are built and configured and I'm working on all the setup.  There may be some minor site issues over the next few days as parts will be moved over but I will limit that as much as possible.

I'm hoping to have everything back to 100% by the end of the week.

Thanks for your patience as I work through this upgrade.
Quote
07-29-2017, 07:49 AM, (This post was last modified: 07-31-2017, 04:33 PM by skeight.)
#3
Posts: 1,163
Threads: 116
Joined: Feb 2011
Reputation: 45
RE: Infrastructure Migration
skeight Offline Administrator View worlds
********
Progress is moving along smoothly and a big part of the server migration is complete.

I will be turning off uploads on July 29th July 30th in the evening sometime (4pm CST ~ish). 
Do not edit your worlds during this downtime. You can edit worlds again

This will let me do some final database/file syncs and migrate the main website to the new server.  There are still some steps to complete when that is done but I will update as the process continues.
Quote
07-31-2017, 04:33 PM, (This post was last modified: 07-31-2017, 09:31 PM by skeight.)
#4
Posts: 1,163
Threads: 116
Joined: Feb 2011
Reputation: 45
RE: Infrastructure Migration
skeight Offline Administrator View worlds
********
We're functioning on the new servers!  Waiting on DNS entries to propagate for everyone to swtich over but if you're reading this you're on the new servers!

Still migrating map files and currently working on getting uploading back up and running.

Thanks again for your patience through this long migration.  This is a side project for me so it's a ton of extra work to make this migration to provide better sustainability for the future of MCWM.
Quote
07-31-2017, 09:31 PM,
#5
Posts: 1,163
Threads: 116
Joined: Feb 2011
Reputation: 45
RE: Infrastructure Migration
skeight Offline Administrator View worlds
********
Uploading is enabled - worlds will be placed in the queue to be processed!

The next step is migrating the actual map images over and getting the new render process working.  The transferring of the map files is taking an extremely long time.  This site has about 16TB worth of map images which equates into hundreds of millions of really small files.

I've been doing the math and I believe it will be way quicker to just re-render the maps than continue to transfer all the files.  Given the current time frame I am looking at about another month of transferring files and that doesn't even include syncing up any changes when it does finally finish.

There will be a period of time where most maps will appear blank.
Quote
08-03-2017, 07:29 AM,
#6
Posts: 1,163
Threads: 116
Joined: Feb 2011
Reputation: 45
RE: Infrastructure Migration
skeight Offline Administrator View worlds
********
I just wanted to give everyone an update on the progress.  FYI - it takes *forever* to copy hundreds of millions of the map image files and adjust permissions so they are readable.

Unfortunately, until this is complete I can't turn rendering back on or I risk the integrity of all the data.  I am continuing to copy files and change permissions and it's difficult to give an ETA.

Thank you for your continued patience while the migration continues.
Quote
08-05-2017, 06:51 PM, (This post was last modified: 08-07-2017, 07:18 PM by Zamzam_Uchiha.)
#7
Posts: 2
Threads: 0
Joined: Aug 2017
Reputation: 0
RE: Infrastructure Migration
Zamzam_Uchiha Offline Wood Chopper View worlds
so that's why I can't download Real Life Minecraft Map
Quote
08-08-2017, 08:37 AM,
#8
Posts: 1,163
Threads: 116
Joined: Feb 2011
Reputation: 45
RE: Infrastructure Migration
skeight Offline Administrator View worlds
********
(08-05-2017, 06:51 PM)Zamzam_Uchiha Wrote: so that's why I can't download Real Life Minecraft Map

No, downloads should be functioning as per normal but I do see a problem with this download.  I will investigate.

Update: I am in the process of copying the final files (I can't even make a guess on the ETA for this though) and then I can turn on the rendering again.  I will start with what is in the queue and when that is done I will start adding some older maps to the queue to get them processed again.
Quote
08-10-2017, 03:45 PM,
#9
Posts: 2
Threads: 0
Joined: Aug 2017
Reputation: 0
RE: Infrastructure Migration
Zamzam_Uchiha Offline Wood Chopper View worlds
(08-08-2017, 08:37 AM)skeight Wrote:
(08-05-2017, 06:51 PM)Zamzam_Uchiha Wrote: so that's why I can't download Real Life Minecraft Map

No, downloads should be functioning as per normal but I do see a problem with this download.  I will investigate.

Update: I am in the process of copying the final files (I can't even make a guess on the ETA for this though) and then I can turn on the rendering again.  I will start with what is in the queue and when that is done I will start adding some older maps to the queue to get them processed
Thanks Mate...
I Wish YOU All The Best
Quote
08-13-2017, 03:00 AM,
#10
Posts: 1
Threads: 0
Joined: Oct 2016
Reputation: 0
RE: Infrastructure Migration
Va11Jr Offline Wood Chopper View worlds
When do you finish processing of copy final files, how many days do you finish?
Quote


Forum Jump:


Users browsing this thread: 4 Guest(s)