Building a Magento Cluster
I wanted to upgrade our Magento 1 web server from a simple single beefy a server running Centos6, Apache, Mysql, PHP-FPM and Redis on the same instance to a cluster of servers, with the objective to improve performance, add redundancy and enable easy scaling.
On one hand I could just grab a Cloudways instance and when I need more cpu's just move a magic slider. However I wanted more control and to see what kind of performance I could get splitting out the services.
This setup will be copied for M2 but probably with some changes to caching.
Some Research
The best practice Adobe setup is overkill for what I want.
Although old this Stack Exchange is a good read along with the GitHub docs on building out a big cluster
DSG
262588213843476
Most of the articles out there are older and geared towards data centres and less cloudy, whereas I want to do this fully in Google Cloud Platform. I have moved away from AWS due to better pricing on GCP, and I also find GCP easier and nicer to use.
For phase one I am not going to bother with a load server to split into multiple app servers, but instead split the admin out onto a separate server. So there will be two Compute Engine instances running Debian & NGINX for Magento and the admin panel, a Cloud SQL instance to run the database, a Memorystore instance to run the Redis cache for sessions, and a Filestore instance for the media folder.
On paper I think this looks like a really powerful setup and much simpler than the recommended from Adobe.
- I don't need a slave database as I can have Cloud SQL with high availability in multiple zones with a single checkbox.
- I am not doing to bother with another cache in front yet but try and use Cloudflare caching.
It's also nice to have a bucket that I use for moving sql dumps around.
Phase two I can add more Magento servers and add a cache and load balancer in front of everything.
Getting Started
I signed up for a new .dev domain with Google Domains, attached the domain to Cloudflare and created a project in Google Cloud Platform.
I set the name servers in Google Domains to Cloudflare
Because it's a dev domain HSTS must be turned on. I went through Cloudflare turning on strict SSL, redirecting to HTTPS and DNSSEC and also added SPF records.
Google Cloud Platform
I then created a new project in GCP and started adding the services I needed.
Bucket
I then created a new private bucket to move the database dump and media folder from the old server to GCP
Compute Engine
I created a new instance with decent specs on the EPYC platform.
Cloud SQL
And a new database with high availability.
And set it to private IP so it is only accessible inside the Google VPC.
Then I could create and empty database and import the sql dump I had put into the bucket.
Installing NGINX & PHP
While the database is importing I can go back to the VM and setup NGINX, PHP and pull over the Magento repo and media folder.
I won't bother setting up SSH keys in my WSL Debian install and just use the web SSH for the moment.
sudo apt update && sudo apt upgrade
sudo apt install nginx php-fpm