How to disable excessive resource usage alert emails?

If you have CSF firewall installed in cPanel you might noticed that you’re getting an email every time some process (usually a php script) uses more than 250MB of memory (default value) or more than 1800 secoonds (also default value).

These emails are being sent by LFD service that sends excessive resource usage alerts to the email address which is assigned to it, normally to root user account. This notification points out a particular process or service using excessive server resources and helps in identifying the resource eating process/service. We can either kill/stop the process/service to free the resource or allocate more resource to it, if necessary, increase the limits or simply do nothing and wait for it to finish it’s work.

Sample LFD email message when memory is exceeded:

—Time: Mon Nov 14 09:41:10 2016 +0530
—Account: xxxxxx
—Resource: Virtual Memory Size
—Exceeded: 205 > 200 (MB)
—Executable: /usr/bin/php
—Command Line: /usr/bin/php /home/xxxxxx/public_html/index.php
—PID: 26953 (Parent PID:24974)
—Killed: No

Sample LDF email message when execution time is exceeded:

—Time: Mon Nov 14 09:41:10 2016 +0530
—Account: xxxxxx
—Resource: Virtual Memory Size
—Exceeded: 125389 > 1800 (seconds)
—Executable: /usr/bin/php
—Command Line: /usr/bin/php /home/xxxxxx/public_html/index.php
—PID: 28429 (Parent PID:26561)
—Killed: No

Getting Started
1) Login to your WHM
2) Go to Home >> select Plugins
3) Click “ConfigServer Security & Firewall”
4) Locate and click at “Firewall Configuration” button

Method 1
This method will permanently disable the LFD excessive resource usage alert and you won’t receive any more emails. Performing this method could pose a a potential security/stability issue (you have been warned! 🙂 ).

5) Modify the value of directives PT_USERMEM and PT_USERTIME to 0.

PT_USERMEM = 0
PT_USERTIME = 0

Method 2
We will increase the values of both memory and time to stop or minimize the LFD alerts. If any process/service uses more resources than defined, you will still continue to receive the LFD alerts, but hopefully less.

5) Modify the value of directives PT_USERMEM and PT_USERTIME to desired.

PT_USERMEM = 500
PT_USERTIME = 150000

Finish
Save the changes you have made and restart CSF+LFD. By doing any of these two methods above should result in getting less or no email alerts from LFD.


How to instal s3fs-fuse on CentOS 6.8?

In case you have CentOS 6.8 server (or DigitalOcean droplet/vps) and you want to install s3fs-fuse the proposed way you’ll run into a problem: the s3fs-fuse requires fuse library version 2.8.4 or higher, but your yum install fuse fuse-devel will install you fuse 2.8.3. If you are running CentOS 7 you won’t have this problem at all, since by default, yum will install fuse 2.9.2.

To fix this problem you need to remove the fuse and install it manually.

1. Remove fuse installed by yum

yum remove fuse fuse* fuse-devel

2. Install some dependencies (for both fuse and s3fs-fuse)

yum install automake gcc-c++ git libcurl-devel libxml2-devel make openssl-devel

3. Download and install latest fuse library (version 2.9.7)

cd /usr/local/src
wget https://github.com/libfuse/libfuse/releases/download/fuse-2.9.7/fuse-2.9.7.tar.gz
tar -xzvf fuse-2.9.7.tar.gz
rm -f fuse-2.9.7.tar.gz
mv fuse-2.9.7 fuse
cd fuse/
./configure --prefix=/usr
make
make install
export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig/
ldconfig

4. Test that fuse is installed (should return “2.9.7″)

pkg-config --modversion fuse

5. Install the s3fs-fuse (using the default instructions from github)

git clone https://github.com/s3fs-fuse/s3fs-fuse.git
cd s3fs-fuse
./autogen.sh
./configure
make
sudo make install

6. If there were no errors along the way, this should return the s3fs-fuse version (currently 1.80)

s3fs --version

Hopefully, this will help you successfully install s3fs-fuse on your system. For details on how to use it please refer to documentation at the github page. Please do not hesitate to leave your comments below! Thank you.


Installig git on CentOS 6 fails with Requires: libcurl.so.3()(64bit)

Today I had to install git on one server running on CentOS 6, but yum install git returned the following error:

Error: Package: git-1.7.12.4-1.el5.rf.x86_64 (rpmforge)
Requires: libcurl.so.3()(64bit)

To fix this bug you have to run this command:

and then confirm the removal of that package:

Loaded plugins: fastestmirror, security
Setting up Remove Process
Resolving Dependencies
--> Running transaction check
---> Package rpmforge-release.x86_64 0:0.5.3-1.el5.rf will be erased
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================================================================================================================================================================================
 Package                                                       Arch                                                Version                                                      Repository                                              Size
=============================================================================================================================================================================================================================================
Removing:
 rpmforge-release                                              x86_64                                              0.5.3-1.el5.rf                                               @rpmforge                                               13 k

Transaction Summary
=============================================================================================================================================================================================================================================
Remove        1 Package(s)

Installed size: 13 k
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Erasing    : rpmforge-release-0.5.3-1.el5.rf.x86_64                                                                                                                                                                                    1/1
  Verifying  : rpmforge-release-0.5.3-1.el5.rf.x86_64                                                                                                                                                                                    1/1

Removed:
  rpmforge-release.x86_64 0:0.5.3-1.el5.rf

Complete!
[root@hosted-by boot]#

After that the standard yum install git will work just fine!


Laravel 5.4 migration: Specified key was too long error solution

Three days ago Laravel 5.4 was released, so I have decided to give it a try. I have made a new Laravel 5.4 project and just when I was about to run the default auth migrations that are shipped with the Laravel almost without any other changes, I got these two red errors:

[Illuminate\Database\QueryException]
SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 767 bytes (SQL: alter table `users` add unique `users_email_unique`(`email`))

 

[PDOException]
SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 767 bytes

What the hell is going on!?!? I have checked the migrations file and everything was okay there unchanged compared to Laravel 5.3. But, according to Laravel Documentation one thing changed:

Laravel 5.4 uses the utf8mb4 character set by default, which includes support for storing “emojis” in the database. If you are upgrading your application from Laravel 5.3, you are not required to switch to this character set.

And, according to their documentation the problem is that I’m running version of MySQL prior to 5.7.7 or MariaDB prior to 10.2.2 (my case). In case you are running versions of MySQL or MariaDB greater than these than most probably that you won’t have this problem at all.

In case you are having this problem – here are the possible solutions:

1. Upgrade MariaDB / MySQL

To update MariaDB on Mac using Brew, first you need to unlink the current one using:

brew unlink mariadb

and then install a dev one using

brew install mariadb --devel

That will install MariaDB version 10.2.3 that works fine. Remember that after installation is done you need to  stop/start the service:

brew services stop mariadb
brew services start mariadb

2. Stick with utf8

In case you don’t want to use the utf8mb4 (that is now setup by default in Laravel 5.4) – instead you want to stick with utf8 that was in use in Laravel 5.3 and all versions before, simply edit /config/database.php  and find these two lines in mysql driver:

'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',

and replace them with with

'charset' => 'utf8',
'collation' => 'utf8_unicode_ci',

That will stop using the utf8mb4 and will change back to utf8. The good thing about this is that you’ll be able to fit 25% more characters into the same columns in database compared with utf8mb4 and that you’ll be able to use existing databases and projects without the need of converting them if you don’t want to. And the downside is that some characters will not be able to be saved to database like this mac command character ⌘ or some emojis. Utf8 can only store Plane Unicode while utf8mb4 can store any unicode character.  Utf8mb4 is also 100% backwards compatible with utf8.

3. Use utf8mb4 without MySQL/MariaDB upgrade

In case you want to use utf8mb4 but you don’t want to or simply can’t upgrade your MySQL/MariaDB there is a simple solution  proposed by Laravel documentation. All you need to do is edit AppServiceProvider located at App\Providers\AppServiceProvider.php and add the following line to the boot() method and load the Schema facade:

use Illuminate\Support\Facades\Schema;

/**
 * Bootstrap any application services.
 *
 * @return void
 */
public function boot()
{
    Schema::defaultStringLength(191);
}

What that will do is actually change the default maximum field length in the database and actually shorten your maximum string length from 255 to maximum of 191 characters (utf8 uses 3 bytes per character while utf8mb4 uses 4 bytes per character your field can now hold 25% less characters 255 * 75% = 191.25). So if you don’t set the string field by hand in the migration the new default will be 191. You can increase the length if you need it manually by defining it:

$table->string('name', 255);

But this will not be possible on indexed fields since these can’t be longer than 191 characters. So keep this in mind in case you’re planing to use utf8mb4.

In case you want to convert the database from utf8 to utf8mb4 for any existing projects or if you are upgrading from Larvel 5.3 or older to Laravel 5.3 this is the code you need to run for every database, table and column:

Make sure you have the database backups before making these changes!!!!

# For each database:
ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;

# For each table:
ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

# For each column:
ALTER TABLE table_name CHANGE column_name column_name VARCHAR(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

# (Don’t just blindly copy-paste this! The exact statement depends on the column type, maximum length, and other properties. The above line is just an example for a `VARCHAR` column.)

Once again: when converting from utf8 to utf8mb4, the maximum length of a column or index key is unchanged in terms of bytes but since utf8mb4 uses 25% more bytes per character your columns can now accept less characters. Therefore, it is smaller in terms of characters, because the maximum length of a character is now four bytes instead of three.

For example, a TINYTEXT column can hold up to 255 bytes, which correlates to 85 three-byte or 63 four-byte characters. Let’s say you have a TINYTEXT column that uses utf8 but must be able to contain more than 63 characters. Given this requirement, you can’t convert this column to utf8mb4 unless you also change the data type to a longer type such as TEXT — because if you’d try to fill it with four-byte characters, you’d only be able to enter 63 characters, but not more.

The same goes for index keys. The InnoDB storage engine has a maximum index length of 767 bytes, so for utf8 or utf8mb4 columns, you can index a maximum of 255 or 191 characters, respectively. If you currently have utf8 columns with indexes longer than 191 characters, you will need to index a smaller number of characters when using utf8mb4.

If you plan on using utf8mb4 make sure to setup your MySQL server accordingly by editing /etc/my.cnf file accordingly:

[client]
default-character-set = utf8mb4

[mysql]
default-character-set = utf8mb4

[mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci

Conclusion

If your application won’t use any special characters or emojis or languages such as Chinese, Japanese, and Korean you’re most probably fine sticking with utf8. But if you can – you should move to utf7mb4 since that will stop you from loosing 4-byte characters when a user ads these in comments or in message or whenever you store these in your database. You should always strive to full unicode support everywhere in your apps and updating your database and code might take some time but it’s definitely worth the time and effort.


How to block language spam

While I was working on my referral traffic spam article today, I have noticed one more form of spam in my Google Analytics statistics. It’s called a language spam and in this case it had something to do with Trump LOL.

language-spam

Here’s the language spam volume:

language-spam-volume

When I filter it by country I can again see that these bots are coming from Russia (same as with referral spam). Now, apparently, someone from Russia (actually the guy behind this is Vitaly Popov, a well-known Analytics spammer) wants me to open this url: secret.ɢoogle.com and to vote for Donald Trump ?!?!? Mind blowing, really… As, you can clearly see that it’s again multi-byte domain, and that the letter G is multi-byte letter, and that it’s clearly a fake url/domain so don’t open it!

This spam actually just hits your Google Analytics, and no bot traffic is sent to your site, so all you need to do is to simply filter out these fake languages. To make things perfectly clear this kind of spam can’t harm your SEO or Rankings because there are no real backlinks or traffic to your site and Google also deosn’t use data from your Google Analytics for website ranking. The only one this spam is affecting is you! You are the only one who can see it while you’re checking your Google Analytics, and you should block it so your statistics are accurate and you can make some good decisions based on them.

Here’s the best way how to block the language spam: go to Google Analytics, and click Admin. Now under your account click All Filters:

and then click on Add filter and make a filter like this (follow my arrows from top to bottom of the picture):

  • Filter Name: write something descriptive so you know what’s this filter for: blocks language spam is a good name
  • Filter Type: pick Custom
  • Filter Field: search for and pick Language Settings
  • Filter Pattern: enter \.
  • Apply Filter to Views: select views that you want to apply this filter on and click Add (moves them from left to right into selected views list)
  • Click Save

That’s it! You have created a filter that will discard any language containing a dot (basically a domain name). This should remove your language spam from future Google Analytics stats.

However if you would like to filter current data and see the correct stats without the language spam hits you’ll have to create a new segment like this: on just about any report in Google Analytics click on +Add Segment:

and now click on red button +New Segment:

Now enter some meaningful name, for the language pick does not contain and just add a single dot in the filed right to it and press save above:

Here you can see the result of this filter with before and after charts:

This way you can filter data on any report. Just click add segment and search for the new segment we created (so it’s important to add a meaningful name to the segment so you can easily locate it anytime you need it).

 

 


CodeIgniter white page on cPanel

After cPanel upgrade from EasyApache3 to EasyApache4 all my CodeIgniter installations just broke on that server, returning just a white page without any error message. I spent more than one hour trying to figure out what the problem is. While testing out I noticed empty files named like these “0bce68a7a37e584ace98f0fd242a237d1662613e” or “296eed9bfb533552b0c3a9c8fdd784067eea216d”   (without any extension) started popping in my project folder. I figured out that these must be session paths so something is wrong with that.

I tried enabling php error logging but it didn’t help since it didn’t log anything since all code is right and it was working before but it just stopped after I started using EasyApache4.

So I went and enabled CodeIgniter logging that proved to be a better idea. To enable it open config.php for editing, locate

$config['log_threshold'] = 0;

And change that value to 4. That will enable internal logging system and logs will be saved in /application/logs folder. What I found out just conf

ERROR - 2016-11-29 22:36:29 --> Severity: Warning --> mkdir(): Invalid path Session_files_driver.php 117
ERROR - 2016-11-29 22:36:29 --> Severity: error --> Exception: Session: Configured save path '' is not a directory, doesn't exist or cannot be created. Session_files_driver.php 119

My session save path at config.php was set to null:

$config['sess_save_path'] = NULL;

Once I got it changed to default (alternatively you can set it up to any other folder you want locally, so you can have full control over your sessions):

$config['sess_save_path'] = sys_get_temp_dir();

everything started working again! Now this is second time I have similar quiet errors with CodeIgniter where it reports absolutely nothing – instead it just gives white screen and this can be really frustrating to debug.

 


	
			
		

How to install Git on cPanel/WHM server running CentOS

Today I wanted to install Git on a server with cPanel/WHM and have got the following result:

yum install git
Resolving Dependencies
--> Running transaction check
---> Package git.x86_64 0:1.7.1-4.el6_7.1 will be installed
--> Processing Dependency: perl-Git = 1.7.1-4.el6_7.1 for package: git-1.7.1-4.el6_7.1.x86_64
--> Processing Dependency: perl(Git) for package: git-1.7.1-4.el6_7.1.x86_64
--> Processing Dependency: perl(Error) for package: git-1.7.1-4.el6_7.1.x86_64
--> Finished Dependency Resolution
Error: Package: git-1.7.1-4.el6_7.1.x86_64 (base)
           Requires: perl-Git = 1.7.1-4.el6_7.1
Error: Package: git-1.7.1-4.el6_7.1.x86_64 (base)
           Requires: perl(Git)
Error: Package: git-1.7.1-4.el6_7.1.x86_64 (base)
           Requires: perl(Error)
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

Then I did some research and have found out that Gis is already installed on cPanel as of version 11.36 but the problem is – it’s installed on this path:

/usr/local/cpanel/3rdparty/bin/git

So for ease of use I advise you to make a simple symlink:

ln -s /usr/local/cpanel/3rdparty/bin/git /usr/bin/git

so you don’t have to remember the path above and you can simply use it like you got used.

Note: In order for user to be able to login to SSH, it must be allowed in WHM first at Account Functions > Manage Shell Access and there  just switch from Disabled Shell to Jailed Shell.

Happy Gitting!


How to remove old kernels on RHEL/CentOS

If your /boot partition is running out of disk space this is probably because you got bunch of old Linux kernels that got stockpiled there and are no longer in use or needed for system to function normally. If you’re running a WHM/cPanel you’ll get an email with subject:  DISKWARN blocks: Mount Point “/boot” 

Usually on a system update, if there is a new Linux Kernel it will get downloaded and initialized while the last one will be kept for just in case the first one fails for any reason. In couple of months or years you could already have 5-6 of them while only the latest Linux kernel would be in use and the previous one will be kept as  a backup. Removing these by hand is possible but an mistake could end up with a system that won’t boot.

To prevent that and free up disk space at /boot partition on Red Hat Linux or CentOS Linux all you need to do is simply to execute this command:

sudo yum install yum-utils && sudo package-cleanup --oldkernels --count=2

This will remove all your old kernels keeping just the last two (latest one and the previous one just in case) so you won’t get these emails or notifications anymore and you’ll keep your /boot partition with enough free space for a new kernel to be installed once released.


Jetpack JSON API failed to activate bug solution

So, this is actually the part 2 of my problems concerning DigitalOcean WordPress 1-click-app and JetPack (about the first issue you can read here). If you read my first post you found out what I had to do in order to activate the Jetpack. All seemed fine and I could see the stats from WordPress installed on DigitalOcean droplet on my WordPress.com account and even some of the active things were working (like Photon or Contact Form).

But, when I wanted to turn on some of the Jetpack modules that I have previously turned off (or that were turned off by default) it didn’t work. For example I wanted to turn off and then turn back on JSON API at General settings tab. When I turned it off it looked like it “worked” (no error messages) but when I wanted to turn it back on I was getting following error message (and the slider won’t go to active position):

JSON API failed to activate. SyntaxError: Unexpected toket < in JSON at position 0

jetpack-json-api-bug

When trying to activate some other Jetpack module I would also get this error:

SyntaxError: JSON Parse error: Unrecognized token ‘<‘” error

I was also unable to disconnect the Jetpack using the WordPress.com account and this was only possible if I would disable Jetpack at plugins and re-enable it again. Then I would have to re-connect it with my WordPress account and it’s a loop but the results were the same. It didn’t work!

I have contacted Jetpack support but their help was similar to Microsoft support (restart the computer and try again), or in Jetpack case: disable all plugins and try reconnecting again. Since Jetpack support was unable to help me after we have exchanged couple of emails I have decided to dig deeper into this on my own. DigitalOcean community also wasn’t helpful since I found just one comment about someone having the same problem as I do but there were no reply or solution.

So I have opened Chrome debugger and have noticed that when I try to activate the JSON API it actually tries to make a POST call to url: /wp-json/jetpack/v4/module/json-api/activate:

wp-json-jetpack-404

and the result is 404 – the standard 404 page! Since there files on that path and WordPress installation doesn’t have wp-json folder at all I figured out that this is mod_rewrite path so I knew in what direction to focus my exploration. I’d like to mention here that this also happened on brand new, zero days blogs, no plugins installed, default theme: basically on out of the box WordPress installations. Also permalinks worked so mod_rewrite was working normally (or it appeared to be working normally), and that made the situation even more confusing.

Out of pure despair I tried re-enabling mod_rewrite and restarting apache:

sudo a2enmod rewrite
sudo service apache2 restart

And it FU*KIN WORKED!!! 

I have tired this on my other 5 or 6 sites hosted on DigitalOcean and it worked on all of them! I hope that some one from Jetpack or some geeks from DigitalOcean will see this post. I’d love to hear more about this issue. How is possible that mod_rewrite is working without being enabled on Apache or if it’s enabled why it doesn’t work for Jetpack plugin? I’m puzzled and I’d like some explanation.

Also I have one more bug that I don’t know how to solve. When I login to these WordPress admins I have this error on every (admin) page I load in my debugger:

Uncaught TypeError: Cannot read property ‘type’ of undefined(…)

This traces back to https://widgets.wp.com/notifications/webpack:/webpack/bootstrap_905f… as you can see in detail on this screenshot:

cannot-read-property-type-of-undefined-bootstra-bug

Update: 05th December 2016
Now Jetpack is works normally on my DigitalOcean droplets, but on some occasions while setting up the Jetpack options I saw this message: Warning! Multibyte support missing!

wp-multibyte

This means that multi-byte support php module was not installed. To get it installed simply install package php-mbstring using apt-get (or yum in case you’re using CentOS/RHEL Linux), restart Apache and you’re good to go!

sudo apt-get install php-mbstring
sudo service apache2 restart

PHP multibyte should now be installed and these messages should no longer appear.


Images won’t center align in WordPress

If you click on the image and hit the Align Center icon inside the editor, it will give to that image a class named “aligncenter” (or left “alignleft” and right to be “alignright” in case you align it left or right) and it will look good in the editor. In some cases once you save the changes and go to check how the image looks on the site you can notice that it’s not properly aligned and instead of being centered your image is right aligned.

This issue occurs on some themes while it works fine on the other (it works nice on the default WordPress theme) and the reason for that is that the theme doesn’t define these aligncenter (or alignleft and alignright) classes in CSS. These are introduced with WordPress 2.5 so some older themes are usually lack this and thus don’t work as expected.

There are some workarounds to solve this issue, but the best one is to just edit default styles.css and add the following css classes in it:

/* =WordPress Core
-------------------------------------------------------------- */
.alignnone {
    margin: 5px 20px 20px 0;
}

.aligncenter,
div.aligncenter {
    display: block;
    margin: 5px auto 5px auto;
}

.alignright {
    float:right;
    margin: 5px 0 20px 20px;
}

.alignleft {
    float: left;
    margin: 5px 20px 20px 0;
}

a img.alignright {
    float: right;
    margin: 5px 0 20px 20px;
}

a img.alignnone {
    margin: 5px 20px 20px 0;
}

a img.alignleft {
    float: left;
    margin: 5px 20px 20px 0;
}

a img.aligncenter {
    display: block;
    margin-left: auto;
    margin-right: auto
}

.wp-caption {
    background: #fff;
    border: 1px solid #f0f0f0;
    max-width: 96%; /* Image does not overflow the content area */
    padding: 5px 3px 10px;
    text-align: center;
}

.wp-caption.alignnone {
    margin: 5px 20px 20px 0;
}

.wp-caption.alignleft {
    margin: 5px 20px 20px 0;
}

.wp-caption.alignright {
    margin: 5px 0 20px 20px;
}

.wp-caption img {
    border: 0 none;
    height: auto;
    margin: 0;
    max-width: 98.5%;
    padding: 0;
    width: auto;
}

.wp-caption p.wp-caption-text {
    font-size: 11px;
    line-height: 17px;
    margin: 0;
    padding: 0 4px 5px;
}

/* Text meant only for screen readers. */
.screen-reader-text {
	clip: rect(1px, 1px, 1px, 1px);
	position: absolute !important;
	height: 1px;
	width: 1px;
	overflow: hidden;
}

.screen-reader-text:focus {
	background-color: #f1f1f1;
	border-radius: 3px;
	box-shadow: 0 0 2px 2px rgba(0, 0, 0, 0.6);
	clip: auto !important;
	color: #21759b;
	display: block;
	font-size: 14px;
	font-size: 0.875rem;
	font-weight: bold;
	height: auto;
	left: 5px;
	line-height: normal;
	padding: 15px 23px 14px;
	text-decoration: none;
	top: 5px;
	width: auto;
	z-index: 100000; /* Above WP toolbar. */
}

 

You can add the code anywhere in the file, but the end of the file is probably your safe bet. Once you save the changes you can reload the site and your images should now be aligned same way as in your editor.