How to clear junk file from system Kali Linux

The first way

sudo apt-get clean

to clear the cache file

sudo apt-get autoclean

to clear the cache file automatically

sudo apt-get autoremove

To clean file / dependencies that are not needed from the system.

  

                                                                       The second way

                                                                         with Bleachbit

                          sudo apt-get install bleachbit




                                                     Download: bleachbit_1.10_all_debian8.deb

Home users vs Servers

There are two types of Linux users:

1. Home Users – i.e. Desktop
2. Servers – i.e. WebServers

For Home users, duh! do whatever you want … you can just do much damage. You clean cache, and the system will be slowed for about 3 seconds and then re-populate memory with necessary files. You can safely run echo 3 to cleanup maximum memory:

sync; echo 3 > /proc/sys/vm/drop_caches

For Servers, (i.e. WebServer, A VPS with Low memory), it is much safer to use option 1, cleaning pagecaches only.

sync; echo 1 > /proc/sys/vm/drop_caches

Why? cause pagecaches fills up massive part of your servers memory and in case of a Apache webserver, 50% of memory is used for pagecaches.

Example scenarios

You need to run free -m and drop_caches to see the differences:

My Desktop

Run the following command to see how much memory is being committed:

free -m

Then run

sync; echo 3 > /proc/sys/vm/drop_caches

As you can see, my desktop was doing next to nothing, so there’s not much to cleanup.

My server

But because my server is always busy and it’s got a small memory pool (for a server), running drop_caches makes a massive difference:

super@myserver [~]# 
super@myserver [~]# free -m
             total       used       free     shared    buffers     cached
Mem:         12792      12151       1640          0       1177      11263
-/+ buffers/cache:       1711      12080
Swap:        11999       1131      11868
super@myserver [~]# 
super@myserver [~]# sync; echo 1 > /proc/sys/vm/drop_caches
super@myserver [~]# 
super@myserver [~]# free -m
             total       used       free     shared    buffers     cached
Mem:         12792       1831      11960          0          0       1132
-/+ buffers/cache:        697      12094
Swap:        11999       1131      11868
super@myserver [~]# 

How awesome is that? In an instant I recovered a massive pool of memory and my terminal to the server became lot more responsive.

Use cron to drop caches regularly

Its a good idea to schedule following in crontab to automatically flushing cache on regular interval.

Option 1: drop_caches using cron

Edit crontab file and add the following line in red at the end of it.

# crontab -e
0 4 * * * sync; echo 1 > /proc/sys/vm/drop_caches

The command sync; echo 1 > /proc/sys/vm/drop_caches
will execute every minute of 4am every daywill execute at 4:00am every day.

Option 2: drop_caches using sysctl

Alternative you can use also sysctl to change the drop_caches value:

# crontab -e
0 4 * * * sync; /sbin/sysctl vm.drop_caches=1

Option 3: drop_caches using a bash script

Create a shell script say clearcache.sh under root partition and enter following contents

#!/bin/sh
sync; echo 1 > /proc/sys/vm/drop_caches

Now, set the permission of script /root/clearcache.sh to 755

chmod 755 /root/clearcache.sh

Now edit crontab file

crontab -e

Enter following line to set cronjob for clearing cache everyday at 4am

0 4 * * * /root/clearcache.sh

Restart crond service

Restart crond service for each options (you only need to choose any of the options above)

/etc/init.d/crond restart

Performance gain and supporting graphs

There seems to be lot of posts and questions around whether this method should be used or not. Here’s my scenario/explanation:

I’ve struggled months with Linux Kernel panic issue: hung_task_timeout_secs and blocked for more than 120 seconds and tweaking vm.dirty_ratio and vm.dirty_backgroud_ratio allowed my VPS to function normally for few months. Then traffic increased again (and sudden bursts of traffic from twitterand reddit) and I couldn’t afford to increase RAM anymore. This kernel hack (be that may outside of recommended usage) allowed me to use cron and Monit to

1. Clear pagecache,
2. Free up enough memory to restart HTTPD automagically.

without having to do a cold restart. I am not claiming this a perfect solution but if used with ‘sync‘ and  if database optimization is enabled, it is yet to fail me.

Let me put it this way, it is also against recommendation to send a crafted fragmented packet to a server causing an outage, but we’ve all tested our server’s against such attacks.

I don’t think a big organization needs such a hack as for them it’s just easier to upgrade RAM, but for the little guys and VPS owners, this could mean the difference between 90% and 99.9% uptime. (I’ve got 99.7% uptime but that’s my fault, I keep testing in Prod server).

Here’s the RAW logs from today:

someuser@someserver [/logs]# grep 'Jan 14' memcleaner.log
Jan 14  00:10:52 127.0.0.1 PreScript Memory Usage = 85 %
Jan 14  00:10:52 127.0.0.1 PageCache Cleaned
Jan 14  03:02:55 127.0.0.1 PreScript Memory Usage = 97 %
Jan 14  03:02:55 127.0.0.1 PageCache Cleaned
Jan 14  05:42:54 127.0.0.1 PreScript Memory Usage = 86 %
Jan 14  05:42:54 127.0.0.1 PageCache Cleaned
Jan 14  09:02:50 127.0.0.1 PreScript Memory Usage = 87 %
Jan 14  09:02:50 127.0.0.1 PageCache Cleaned
...snipped...

and Monit logs

someuser@someserver [/logs]# egrep 'Jan 14.*trying to restart' monit.log
[Jan 14 03:03:17] info     : 'apache' trying to restart
...snipped...

and lastly here’s sar logs

someuser@someserver [/logs]# sar -r
00:00:01    kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
00:10:02       610068   3214528     84.05    108604   2351084   1696672     28.89
03:00:01       747668   3076928     80.45    162776   2116960   1709460     29.11
03:10:03      1484612   2339984     61.18     60640   1592920   1723404     29.35
03:20:01      1326836   2497760     65.31     73848   1710880   1857400     31.63
05:30:01       785668   3038928     79.46    165852   2161624   1581312     26.93
05:40:01       770060   3054536     79.87    168364   2167852   2104060     35.83
05:50:01      1414824   2409772     63.01     70100   1618396   1830092     31.16
...snipped...

As you can see, PageCache was cleaned 4 times overnight, service was restarted only once when Memory reached 97%. I had a massive spike of traffic (200%) at 3am. Normally, most servers would get stuck at and become unresponsive when memory is 97% or +, refer to the Kernel panic link above. This whole time, sar was just cruising along thinking everything just fine (cause it’s set to every 10minutes to save resources) … We use 8GB+ RAM minimum at work for just reverse proxies, but a VPS is no match for that as it’s doing everything from DNS, DB, Webserver, syslog, sshd, cPanel, backup etc.