Why is my Magento 1 site slow and getting slower?

Hi, sorry to hear your Magento 1 site is getting slower. It’s February 2020 and in just a couple of months Magento 1 is being sunsetted (June 2020). This doesn’t mean your site is going to stop working but it does mean there are lots of things going on in the eCommerce world, and the cyber criminal world that are going to affect you.

What is making my Magento 1 site slower?

Conceptually, a Magento 1 site is just like any other website. The main reasons why a website runs slow is because of:

Not enough resources.
All websites require memory, processing power and storage space. If either of these areas are underpowered then the site is not going to run as fast as it can.

Too much traffic for the resources available.
Similar to not enough resources but traffic can cause peaks and troughs. There is also such a thing as “bad traffic”

Poorly written code.
Poorly written code can contribute to a slow site. If your code is trying to do too many things or doing it in an inefficient way it can use far too much resource that it crucially needs. In simple terms you can also write code that forgets to stop. When this happens processes keep running and running for no reason causing a massive draw on critical memory and processing power.

Reason 1: Sales Bots

We are all looking for our next client but some are a little more unscrupulous than others. We take the route of being experts in our class, providing helpful content and asking happy customers to recommend us. Others are relying on bots to crawl the web to create lists before making a direct approach.

To build these lists companies are using lots of bots, and bots take up customer seats on your website.

If your website can handle 1000 connections at once and you have 1100 trying to connect; 200 of them bots. The 900 real customers on your site will get a slower experience because your website resources are overwhelmed.

Tip: You can check whether your site is getting targeted in Google Analytics.
Visit Google Analytics > Acquisition > All Traffic > Referrals. Check the referral urls for unusual activity.

How to prevent bots slowing down your Magento site

1. Block Bots with Robots.txt

One of the quickest and easiest ways is to update your robots.txt file. This blocks bots that respect robots.txt from accessing your site.
Within your Magento Admin Panel visit: Content > Design > Configuration > Level = Website > Search Engine Robots

Here is an example you could try within the “Edit custom instruction of robots.txt File” input field

Note: If your custom instructions are blank click Reset to Defaults and then you can edit the normal settings.

# Google Image Crawler Setup
User-agent: Googlebot-Image
Disallow:
# Crawlers Setup
User-agent: *
# Directories
Disallow: /404/
Disallow: /app/
Disallow: /cgi-bin/
Disallow: /downloader/
Disallow: /errors/
Disallow: /includes/
#Disallow: /js/
#Disallow: /lib/
Disallow: /magento/
#Disallow: /media/
Disallow: /pkginfo/
Disallow: /report/
Disallow: /scripts/
Disallow: /shell/
Disallow: /skin/
Disallow: /stats/
Disallow: /var/
# Paths (clean URLs)
Disallow: /index.php/
Disallow: /catalog/product_compare/
Disallow: /catalogue/category/view/
Disallow: /catalog/product/view/
Disallow: /cataloguesearch/
#Disallow: /checkout/
Disallow: /control/
Disallow: /contacts/
Disallow: /customer/
Disallow: /customize/
Disallow: /newsletter/
Disallow: /poll/
Disallow: /review/
Disallow: /sendfriend/
Disallow: /tag/
Disallow: /wishlist/
Disallow: /catalog/product/gallery/
# Files
Disallow: /cron.php
Disallow: /cron.sh
Disallow: /error_log
Disallow: /install.php
Disallow: /LICENSE.html
Disallow: /LICENSE.txt
Disallow: /LICENSE_AFL.txt
Disallow: /STATUS.txt
# Paths (no clean URLs)
#Disallow: /*.js$
#Disallow: /*.css$
Disallow: /*.php$
Disallow: /*?SID=

2. Block users by country or IP

Unscrupulous bots are just that. They may not respect the robots.txt file instructions and you’ll need to take a different approach.

Let’s say you are a UK business and only sell to the UK. You only market in the UK and suddenly you are getting lots of traffic from the other side of the world.
Just like in this Magento 1 site.

Let’s say you are a UK business and only sell to the UK. You only market in the UK and suddenly you are getting lots of traffic from the other side of the world.
Just like in this Magento 1 site >>>>>Why so much traffic from Russia, India and Australia? This could be bots – especially if you’re not seeing the usual activities; product visits, purchases and enquiries. Allowing this traffic is currently detrimental to your business and unnecessary as you only sell in the UK. Blocking these countries from accessing the site is likely to deliver a better user experience for your real customers.

Also consider the time of day the traffic is coming. Most UK shoppers will shop in what is considered “normal hours” and on into the evening. If you are seeing high volumes in the early hours and no conversions to orders this is another sign of bots crawling the site.

Reason 2: Cyber Criminals

According to various sources of research around 87% of Magento sites are “High Risk” and left open to cyber attacks.  The biggest cause of this is the basics of cyber security being overlooked, such as simply applying the latest Magento security patches.

In the case of Magento 1 we have known for some time that no more security updates are going to be released after June 2020. In the past cyber criminals would begin attacking sites until the SUPEE patch thwarted their efforts and they would start looking for the next one.

Right now cyber criminals are likely testing Magento 1 sites for vulnerabilities and exploits. Our best (and most pessimistic assumption) is that all of the exploits being discovered by cyber criminals in the Magento 1 codebase and most popular plugins are being hoarded currently. As soon as June 2020 passes the criminals will reveal their hands knowing the masses will not receive any official protective update.

Those who don’t migrate to Magento 2 or another platform will be leaving themselves permanently vulnerable or overwhelmed with complex security development.

How can I see if I am being targeted?

It’s not a foolproof method but we are seeing a number of sites slowing down because they are being hit with malware and bots that are attempting to run mini-programs that open the doors to your website.

To check your Magento 1 site you can look in your nginx access log files or similar depending on your server configuration.

Look for “python”. This is a common coding language, but is also a popular codebase being used to hack sites.

In this example we can see somebody is targeting the site with a script that is attempting to guess the admin login URL. Their first guess is usually the Magento default /admin, then /adminpanel, /controlpanel etc.

85.93.20.30 - - [13/Feb/2020:08:45:28 +0000] "GET /styl-manage HTTP/1.1" 404 4385 "-" "python-requests/2.13.0" "-" 159 9.328 "14238" "9.328" "YOUR-DOMAIN"

85.93.20.66 - - [13/Feb/2020:08:45:30 +0000] "GET /styladmin HTTP/1.1" 404 4378 "-" "python-requests/2.13.0" "-" 157 10.247 "14234" "10.248" "YOUR-DOMAIN"

78.128.113.58 - - [13/Feb/2020:08:45:38 +0000] "GET /sty_admin HTTP/1.1" 404 4380 "-" "python-requests/2.13.0" "-" 157 13.050 "14234" "13.052" "YOUR-DOMAIN"

85.93.20.26 - - [13/Feb/2020:08:45:38 +0000] "GET /styadminpanel HTTP/1.1" 404 4382 "-" "python-requests/2.13.0" "-" 161 13.040 "14242" "13.040" "YOUR-DOMAIN"

85.93.20.34 - - [13/Feb/2020:08:45:39 +0000] "GET /sty_control HTTP/1.1" 404 4382 "-" "python-requests/2.13.0" "-" 159 13.127 "14238" "13.128" "YOUR-DOMAIN"

78.128.113.182 - - [13/Feb/2020:08:45:39 +0000] "GET /stycontrol HTTP/1.1" 404 4381 "-" "python-requests/2.13.0" "-" 158 13.109 "14236" "13.108" "YOUR-DOMAIN"

45.227.253.147 - - [13/Feb/2020:08:45:39 +0000] "GET / HTTP/1.1" 200 7553 "-" "python-requests/2.13.0" "-" 148 13.789 "44226" "13.788" "YOUR-DOMAIN"

78.128.113.30 - - [13/Feb/2020:08:45:39 +0000] "GET /sty-user HTTP/1.1" 404 4382 "-" "python-requests/2.13.0" "-" 156 13.023 "14232" "13.024" "YOUR-DOMAIN"

How to stop Magento Bot attacks?

This article is not intended to go into great detail on how to stop attacks but more to address that lots of attacks happening can slow down your site. So many connections from fake and malicious users running mini programs cause your servers processing power to be diverted from its main purpose. 

If you want to check out some ways to prevent these attacks we would suggest:

• Speaking to your development or hosting team
• Speak to us
• Update your Firewall settings to exclude certain connections and countries
• Consider an IP whitelist and blacklist
• Only allow certain user agents to connect (although these can be spoofed)
• Subscribe to a CDN service like Cloudflare that offers additional security benefits

Summary

• Bots are crawling the internet using up resources to sell you a new site.
• Cyber criminals are pumping your Magento 1 site with scripts to find vulnerabilities.
• You may simply be running out of resources on your current hosting.

Short term fix vs long term fix. What to do next?

The short answer is it’s time to get off of Magento 1. We have a different view to many on this, in that it doesn’t need to be expensive.

Magento 2 is similar in many ways to Magento 1. There are lots of tools available to us to migrate including a Magento Migration Tool built specifically to move people from version 1 to version 2. We use specialist tools for moving catalogues and CMS pages efficiently.

We also have an amazing product called Lumarix. It contains over 200 modules that add enhancements to the barebones of Magento. Apart from some very specific industry requirements these modules cover almost every sales, marketing and customer management wish you might have.

It’s mobile friendly, we can provide a mockup in your brand and style within a few hours and can have most sites migrated within a few weeks.

It doesn’t cost anything to have a chat with us about moving your site, your ambitions for the next few years and put together a short plan and costing specific for you. Why not give us a call?