How To Configure Apache Content Caching on Ubuntu 14.04

What is Caching?

Caching is a method of improving server performance by allowing commonly requested content to be temporarily stored in a way that allows for faster access. This speeds up processing and delivery by cutting out some resource intensive operations.

By creating effective caching rules, content that is suitable for caching will be stored to improve response times, conserve resources, and minimize load. Apache provides a variety of caches suitable for speeding up different types of operations. In this guide, we will be discussing how to configure Apache 2.4 on Ubuntu 14.04 using its various caching modules.

To learn more about developing general caching strategies, check out this article.

An Introduction to Caching in Apache

Apache can cache content with varying levels of sophistication and scalability. The project divides these into three groups according to the method in which the content is cached. The general breakdown is:

• File Caching: The most basic caching strategy, this simply opens files or file descriptors when the server starts and keeps them available to speed up access.
• Key-Value Caching: Mainly used for SSL and authentication caching, key-value caching uses a shared object model that can store items which are costly to compute repeatedly.
• Standard HTTP caching: The most flexible and generally useful caching mechanism, this three-state system can store responses and validate them when they expire. This can be configured for performance or flexibility depending on your specific needs.

A quick look at the above descriptions may reveal that the above methods have some overlap, but also that it may be helpful to use more than one strategy at the same time. For instance, using a key-value store for your SSL sessions and enabling a standard HTTP cache for responses could allow you to take significant load off of your data sources and speed up many content delivery operations for your clients.

Now that you have a broad understanding of each of Apache's caching mechanisms, let's look at these systems in more detail.

File Caching

General Overview

• Primary modules involved: mod_file_cache
• Main use cases: storing either file contents or file descriptors when the server starts. These are static representations that cannot reliably be changed until the server is restarted.
• Features: simple, improves performance of slow filesystems
• Drawbacks: experimental feature, does not respond to updates on the filesystem, must be used sparingly to fit within operating system's limitations, can only be used on static files

The Details

The mod_file_cache module is mainly used to speed up file access on servers with slow filesystems. It provides a choice of two configuration directives, both of which aim to accelerate the process of serving static files by performing some of the work when the server is started rather than when the files are requested.

The CacheFile directive is used to specify the path to files on disk that you would like to accelerate access to. When Apache is started, Apache will open the static files that were specified and cache the file handle, avoiding the need to open the file when it is requested. The number of files that can be opened in this way is subject to the limitations set by your operating system.

The MMapFile directive also opens files when Apache is first started. However, MMapFile caches the file's contents in memory rather than just the file handler. This allows for faster performance for those pages, but it has some serious limitations. It maintains no record of the amount of memory it has used, so it is possible to run out of memory. Also note that child processes will copy any of the allocated memory, which can result in faster resource depletion than you initially may anticipate. Only use this directive sparingly.

These directives are evaluated only when Apache starts. This means that you cannot rely on Apache to pick up changes made after it has started. Only use these on static files that will not change for the lifetime of the Apache session. Depending on how the files are modified, the server may be notified of changes, but this is not expected behavior and will not always work correctly. If changes must be made to files passed to these directives, restart Apache after the changes have been made.

How To Enable File Caching

File caching is provided by the mod_file_cache module. To use this functionality, you'll need to enable the module.

When running Ubuntu 14.04, the module will be installed but disabled when you install Apache. You can enable the module by typing:

sudo a2enmod file_cache

Afterwards, you should edit the main configuration file to set up your file caching directives. Open the file by typing:

sudo nano /etc/apache2/apache2.conf

To set up file handle caching, use the CacheFile directive. This directive takes a list of file paths, separated by spaces, like this:

/etc/apache2/apache2.conf

CacheFile /var/www/html/index.html /var/www/html/somefile.index

When the server is restarted, Apache will open the files listed and store their file handles in the cache for faster access.

If, instead, you wish to map a few files directly into memory, you can use the MMapFile directive. Its syntax is basically the same as the last directive, in that it simply takes a list of file paths:

/etc/apache2/apache2.conf

MMapFile /var/www/html/index.html /var/www/html/somefile.index

In practice, there would be no reason to configure both CacheFile and MMapFile for the same set of files, but you could use both on different sets of files.

When you are finished, you can save and close the files. Check the configuration file syntax by typing:

sudo apachectl configtest

If the last line reads Syntax OK, you can safely restart your Apache instance:

sudo service apache2 restart

Apache will restart, caching the file contents or handlers depending on the directives you used.

Key-Value Caching

General Overview

• Primary modules involved: mod_socache_dbm, mod_socache_dc, mod_socache_memcache,mod_socache_shmcb
• Supporting modules involved: mod_authn_socache, mod_ssl
• Main use cases: storing SSL sessions or authentication details, SSL stapling
• Features: shared object cache to store complex resources, can assist in SSL session caching and stapling, flexible backends
• Drawbacks: has no validation mechanisms, need to configure separate software for more performant/flexible backends, some bugs in code

The Details

Key-value caching is more complex than file caching and has more focused benefits. Also known as a shared object cache, Apache's key-value cache is mainly used to avoid repeating expensive operations involved with setting up a client's access to content, as opposed to the content itself. Specifically, it can be used to cache authentication details, SSL sessions, and to provide SSL stapling.

Note

Currently, there are some issues with every shared object cache provider. References to the issues will be outlined below. Take these into consideration when evaluating whether to enable this feature.

The actual caching is accomplished through the use of one of the shared object caching provider modules. These are:

• mod_socache_dbm: This backend uses the simple dbm database engine, which is a file-based key-value store that makes use of hashing and fixed-size buckets. This provider suffers from some memory leaks, so for most cases it is recommended to use mod_socache_shmcb instead.
• mod_socache_dc: This provider uses the distcache session caching software. This project has not been updated since 2004 and is not even packaged for some distributions, so use with a healthy dose of caution.
• mod_socache_memcache: This uses the memcache distributed memory object cache for storing items. This is the best option for a distributed cache among multiple servers. Currently, it does not properly expire entries, but a patch was committed to the trunk of Apache's version control that fixes the issue.
• mod_socache_shmcb: Currently, this is the best option for key-value caching. This caches to a cyclic buffer in shared memory, which will remove entries as it becomes full. It currently chokes on entries over 11k in size.

Along with the above provider modules, additional modules will be required depending on the objects being cached. For instance, to cache SSL sessions or to configure SSL stapling, mod_ssl must be enabled, which will provide the SSLSessionCache and SSLStaplingCache directives respectively. Similarly, to set up authentication caching, the mod_authn_socache module must be enabled so that theAuthnCacheSOCache directive can be set.

How To Enable Key-Value Caching

With the above bugs and caveats in mind, if you still wish to configure this type of caching in Apache, follow along below.

The method used to set up the key-value cache will depend on what it will be used for and what provider you are using. We'll go over the basics of both authentication caching and SSL session caching below.

Currently, there is a bug with authentication caching that prevents passing arguments to the cache provider. So any providers that do not provide default settings to fall back on will have issues.

Authentication Caching

Authentication caching is useful if you are using an expensive authentication method, such as LDAP or database authentication. These types of operations can have a significant impact on performance if the backend must be hit every time an authentication request is made.

Setting up caching involves modifying your existing authentication configuration (we will not cover how to set up authentication in this guide). The modifications themselves will be much the same regardless of the backend authentication method. We'll use mod_socache_shmcb for our demonstration.

First, enable the authn_socache module and the mod_socache_shmcb provider module by typing:

sudo a2enmod authn_socache

sudo a2enmod socache_shmcb

Open your main Apache configuration file so that you can specify this shared cache backend for use with authentication:

sudo nano /etc/apache2/apache2.conf

Inside, towards the top of the file, add the AuthnCacheSOCache directive. Specify that shmcb should be used as the provider. If the bug discussed earlier preventing option passing is fixed by the time you read this, you can specify a location and size for the cache. The number is in bytes, so the commented example will result in a 512 kilobyte cache:

/etc/apache2/apache2.conf

AuthnCacheSOCache shmcb

# If the bug preventing passed arguments to the provider gets fixed,
# you can customize the location and size like this
#AuthnCacheSOCache shmcb:${APACHE_RUN_DIR}/auth_cache(512000)

Save and close the file when you are finished.

Next, open your virtual host configuration page that has authentication configured. We'll assume you're using the 000-default.conf virtual host config, but you should modify it to reflect your environment:

sudo nano /etc/apache2/sites-enabled/000-default.conf

In the location where you've configured authentication, modify the block to add caching. Specifically, you need to add the AuthnCacheProvideFor to tell it which authentication sources to cache, add a cache timeout with AuthnCacheTimeout, and add socache to the AuthBasicProvider list ahead of your conventional authentication method. The results will look something like this:

/etc/apache2/sites-enabled/000-default.conf

<VirtualHost *:80>

    . . .

    <Directory /var/www/html/private>
        AuthType Basic
        AuthName "Restricted Files"
        AuthBasicProvider socache file
        AuthUserFile /etc/apache/.htpasswd
        AuthnCacheProvideFor file
        AuthnCacheTimeout 300
        Require valid-user
    </Directory>
</VirtualHost>

The above example is for file authentication, which probably won't benefit from caching very much. However, the implmentation should be very similar when using other authentiation methods. The only substantial difference would be where the "file" specification is in the above example, the other authentication method would be used instead.

Save and close the file. Restart Apache to implement your caching changes:

sudo service apache2 restart

SSL Session Caching

The handshake that must be performed to establish an SSL connection carries significant overhead. As such, caching the session data to avoid this initialization step for further requests can potentially skirt this penalty. The shared object cache is a perfect place for this.

If you have SSL already configured for your Apache server, mod_ssl will be enabled. On Ubuntu, this means that an ssl.conf file has been moved to the /etc/apache2/mods-enabled directory. This actually already sets up caching. Inside, you will see some lines like this:

/etc/apache2/mods-enabled/ssl.conf

. . .

SSLSessionCache         shmcb:${APACHE_RUN_DIR}/ssl_scache(512000)
SSLSessionCacheTimeout  300

. . .

This is actually enough to set up session caching. To test this, you can use OpenSSL's connection client. Type:

openssl s_client -connect 127.0.0.1:443 -reconnect -no_ticket | grep Session-ID

If the session ID is the same in all of the results, your session cache is working correctly. Press CTRL-C to exit back to the terminal.

Standard HTTP Caching

General Overview

• Primary modules involved: mod_cache
• Supporting modules involved: mod_cache_disk, mod_cache_socache
• Main use cases: Caching general content
• Features: Can correctly interpret HTTP caching headers, can revalidate stale entries, can be deployed for maximum speed or flexibility depending on your needs
• Drawbacks: Can leak sensitive data if incorrectly configured, must use additional modules to correctly set the caching policy

The Details

The HTTP protocol encourages and provides the mechanisms for caching responses all along the content delivery path. Any computer that touches the content can potentially cache each item for a certain amount of time depending on the caching policies set forth at the content's origins and the computer's own caching rules.

The Apache HTTP caching mechanism caches responses according to the HTTP caching policies it sees. This is a general purpose caching system that adheres to the same rules that any intermediary server would follow that has a hand in the delivery. This makes this system very flexible and powerful and allows you to leverage the headers that you should already be setting on your content (we'll cover how to do this below).

Apache's HTTP cache is also known as a "three state" cache. This is because the content it has stored can be in one of three states. It can be fresh, meaning it is allowed to be served to clients with no further checking, it can be stale, meaning that the TTL on the content has expired, or it can be non-existent if the content is not found in the cache.

If the content becomes stale, at the next request, the cache can revalidate it by checking the content at the origin. If it hasn't changed, it can reset the freshness date and serve the current content. Otherwise, it fetches the changed content and stores that for the length of time allowed by its caching policy.

Module Overview

The HTTP caching logic is available through the mod_cache module. The actual caching is done with one of the caching providers. Typically, the cache is stored on disk using the mod_cache_disk module, but shared object caching is also available through the mod_cache_socache module.

The mod_cache_disk module caches on disk, so it can be useful if you are proxying content from a remote location, generating it from a dynamic process, or just trying to speed things up by caching on a faster disk than your content typically resides on. This is the most well-tested provider and should probably be your first choice in most cases. The cache is not cleaned automatically, so a tool calledhtcacheclean must be run occasionally to slim down the cache. This can be run manually, set up as a regular cron job, or run as a daemon.

The mod_cache_socache module caches to one of the shared object providers (the same ones discussed in the last section). This can potentially have better performance than mod_cache_disk (depending on which shared cache provider is selected). However, it is much newer and relies on the shared object providers, which have the bugs discussed earlier. Comprehensive testing is recommended before implementing the mod_cache_socache option.

HTTP Cache Placement

Apache's HTTP cache can be deployed in two different configurations depending on your needs.

If the CacheQuickHandler is set to "on", the cache will be checked very early in the request handling process. If content is found, it will be served directly without any further handling. This means that it is incredibly quick, but it also means that it does not allow for processes like authentication for content. If there is content in your cache that normally requires authentication or access control, it will be accessible to anyone without authentication if the CacheQuickHandler is set to "on".

Basically, this emulates a separate cache in front of your web server. If your web server needs to do any kind of conditional checking, authentication, or authorization, this will not happen. Apache will not even evaluate directives within <Location> or <Directory> blocks. Note that CacheQuickHandler is set to "on" by default!

If the CacheQuickHandler is set to "off", the cache will be checked significantly later in the request processing sequence. Think of this configuration as placing the cache between your Apache processing logic and your actual content. This will allow the conventional processing directives to be run prior to retrieving content from the cache. Setting this to "off" trades a bit of speed for the ability to process requests more deeply.

How To Configure Standard HTTP Caching

In order to enable caching, you'll need to enable the mod_cache module as well as one of its caching providers. As we stated above, mod_cache_disk is well tested, so we will rely on that.

Enabling the Modules

On an Ubuntu system, you can enable these modules by typing:

sudo a2enmod cache

sudo a2enmod cache_disk

This will enable the caching functionality the next time the server is restarted.

You will also need to install the apache2-utils package, which contains the htcacheclean utility used to pare down the cache when necessary. You can install this by typing:

sudo apt-get update

sudo apt-get install apache2-utils

Modifying the Global Configuration

Most of the configuration for caching will take place within individual virtual host definitions or location blocks. However, enabling mod_cache_disk also enables a global configuration that can be used to specify some general attributes. Open that file now to take a look:

sudo nano /etc/apache/mods-enabled/cache_disk.conf

With the comments removed, the file should look like this:

/etc/apache2/mods-enabled/cache_disk.conf

<IfModule mod_cache_disk.c>
    CacheRoot /var/cache/apache2/mod_cache_disk
    CacheDirLevels 2
    CacheDirLength 1
</IfModule>

The IfModule wrapper tells Apache to only worry about these directives if the mod_cache_disk module is enabled. The CacheRoot directive specifies the location on disk where the cache will be maintained. The CacheDirLevels and CacheDirLength both contribute towards defining how the cache directory structure will be built.

An md5 hash of the URL being served will be created as the key used to store the data. The data will be organized into directories derived from the beginning characters of each hash. CacheDirLevels specifies the number of subdirectories to create and CacheDirLength specifies how many characters to use as the name of each directory. So a hash of b1946ac92492d2347c6235b4d2611184 with the default values shown above would be filed in a directory structure of b/1/946ac92492d2347c6235b4d2611184. Usually, you won't need to modify these values, but it's good to know what they're used for.

Note

If you choose to modify the CacheRoot value, you'll have to open the /etc/default/apache2 file and modify the value of the HTCACHECLEAN_PATH to match your selection. This is used to clean the cache at regular intervals, so it must have the correct location of the cache.

Some other values you can set in this file are CacheMaxFileSize and CacheMinFileSize which set the ranges of file sizes in bytes that Apache will commit to the cache, as well as CacheReadSize andCacheReadTime, which allows you to wait and buffer content before sending to the client. This can be useful if the content resides somewhere other than this server.

Modifying the Virtual Server

Most of the configuration for caching will happen on a more granular level, either in the virtual host definition or in a specific location block.

Open one of your virtual host files to follow along. We'll assume you're using the default file in this guide:

sudo nano /etc/apache2/sites-enabled

In the virtual host block, outside of any location block, we can begin configuring some of the caching properties. In this guide, we'll assume that we want to turn the CacheQuickHandler off so that more processing is done. This allows us up more complete caching rules.

We will also take this opportunity to configure cache locking. This is a system of file locks that Apache will use when it is checking in with the content origin to see whether content is still valid. During the time when this query is being satisfied, if additional requests for the same content come in, it would result in additional requests to the backend resource, which could cause load spikes.

Setting a cache lock for a resource during validation tells Apache that the resource is currently being refreshed. During this time, the stale resource can be served with a warning header indicating its state. We'll set this up with a cache lock directory in the /tmp folder. We'll allow a maximum of 5 seconds for a lock to be considered valid. These examples are taken directly from Apache's documentation, so they should work well for our purposes.

We will also tell Apache to ignore the Set-Cookie headers and not store them in the cache. Doing so will prevent Apache from accidentally leaking user-specific cookies out to other parties. The Set-Cookieheader will be stripped before the headers are cached.

/etc/apache2/sites-enabled/000-default.conf

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    CacheQuickHandler off

    CacheLock on
    CacheLockPath /tmp/mod_cache-lock
    CacheLockMaxAge 5

    CacheIgnoreHeaders Set-Cookie
</VirtualHost>

We still need to actually enable caching for this virtual host. We can do this with the CacheEnabledirective. If this is set in a virtual host block, we would need to provide the caching method (disk orsocache) as well as the requested URIs that should be cached. For example, to cache all responses, this could be set to CacheEnable disk /, but if you only wanted to cache responses under the /public URI, you could set this to CacheEnable disk /public.

We will take a different route by enabling our cache within a specific location block. Doing so means we don't have to provide a URI path to the CacheEnable command. Any URI that would be served from that location will be cached. We will also turn on the CacheHeader directive so that our response headers will indicate whether the cache was used to serve the request or not.

Another directive we'll set is CacheDefaultExpire so that we can set an expiration (in seconds) if neither the Expires nor the Last-Modified headers are set on the content. Similarly, we'll set CacheMaxExpireto cap the amount of time items will be saved. We'll set the CacheLastModifiedFactor so that Apache can create an expiration date if it has a Last-Modified date, but no expiration. The factor is multiplied by the time since modification to set a reasonable expiration.

/etc/apache2/sites-enabled/000-default.conf

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    CacheQuickHandler off

    CacheLock on
    CacheLockPath /tmp/mod_cache-lock
    CacheLockMaxAge 5

    CacheIgnoreHeaders Set-Cookie

    <Location />
        CacheEnable disk
        CacheHeader on

        CacheDefaultExpire 600
        CacheMaxExpire 86400
        CacheLastModifiedFactor 0.5
    </Location>
</VirtualHost>

Save and close your file when you've configured everything that you need.

Check your entire configuration for syntax errors by typing:

sudo apachectl configtest

If no errors are reported, restart your service by typing:

sudo service apache2 restart

Setting Expires and Caching Headers on Content

In the above configuration, we configured HTTP caching, which relies on HTTP headers. However, none of the content we're serving actually has the Expires or Cache-Control headers needed to make intelligent caching decisions. To set these headers, we need to take advantage of a few more modules.

The mod_expires module can set both the Expires header and the max-age option in the Cache-Control header. The mod_headers module can be used to add more specific Cache-Control options to tune the caching policy further.

We can enable both of these modules by typing:

sudo a2enmod expires

sudo a2enmod headers

After enabling these modules, we can go straight to modifying our virtual host file again:

sudo nano /etc/apache/sites-enabled/000-default.conf

The mod_expires module provides just three directives. The ExpiresActive turns expiration processing on in a certain context by setting it to "on". The other two directives are very similar to each other. TheExpiresDefault directive sets the default expiration time, and the ExpiresByType sets the expiration time according to the MIME type of the content. Both of these will set the Expires and the Cache-Control "max-age" to the correct values.

These two settings can take two different syntaxes. The first is simply "A" or "M" followed by a number of seconds. This sets the expiration in relation to the last time the content was "accessed" or "modified" respectively. For example, these both would expire content 30 seconds after it was accessed.

ExpiresDefault A30
ExpireByType text/html A30

The other syntax allows for more verbose configuration. It allows you to use units other than seconds that are easier for humans to calculate. It also uses the full word "access" or "modification". The entire expiration configuration should be kept in quotes, like this:

ExpiresDefault "modification plus 2 weeks 3 days 1 hour"
ExpiresByType text/html "modification plus 2 weeks 3 days 1 hour"

For our purposes, we'll just set a default expiration. We will start by setting it to 5 minutes so that if we make a mistake while getting familiar, it won't be stored on our clients' computers for an extremely long time. When we're more confident in our ability to select policies appropriate for our content, we can adjust this to something more aggressive:

/etc/apache2/sites-enabled/000-default.conf

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    CacheQuickHandler off

    CacheLock on
    CacheLockPath /tmp/mod_cache-lock
    CacheLockMaxAge 5

    CacheIgnoreHeaders Set-Cookie

    <Location />
        CacheEnable disk
        CacheHeader on

        CacheDefaultExpire 600
        CacheMaxExpire 86400
        CacheLastModifiedFactor 0.5

        ExpiresActive on
        ExpiresDefault "access plus 5 minutes"
    </Location>
</VirtualHost>

This will set our Expires header to five minutes in the future and set Cache-Control max-age=300. In order to refine our caching policy further, we can use the Header directive. We can use the merge option to add additional Cache-Control options. You can call this multiple times and add whichever additional policies you'd like. Check out this guide to get an idea about the caching policies you'd like to set for your content. For our example, we'll just set "public" so that other caches can be sure that they're allowed to store copies.

To set ETags for static content on our site (to use for validation), we can use the FileETag directive. This will work for static content. For dynamically generated content, you're application will be responsible for correctly generating ETags.

We use the directive to set the attributes that Apache will use to calculate the Etag. This can be INode,MTime, Size, or All depending on if we want to modify the ETag whenever the file's inode changes, its modification time changes, its size changes, or all of the above. You can provide more than one value, and you can modify the inherited setting in child contexts by preceding the new settings with a + or -. For our purposes, we'll just use "all" so that all changes are registered:

/etc/apache2/sites-enabled/000-default.conf

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    CacheQuickHandler off

    CacheLock on
    CacheLockPath /tmp/mod_cache-lock
    CacheLockMaxAge 5

    CacheIgnoreHeaders Set-Cookie

    <Location />
        CacheEnable disk
        CacheHeader on

        CacheDefaultExpire 600
        CacheMaxExpire 86400
        CacheLastModifiedFactor 0.5

        ExpiresActive on
        ExpiresDefault "access plus 5 minutes"

        Header merge Cache-Control public
        FileETag All
    </Location>
</VirtualHost>

This will add "public" (separated by a comma) to whatever value Cache-Control already has and will include an ETag for our static content.

When you are finished, save and close the file. Check the syntax of your changes by typing:

sudo apachectl configtest

If no errors were found, restart your service to implement your caching policies:

sudo service apache2 restart

Conclusion

Configuring caching with Apache can seem like a daunting job due to how many options there are. Luckily, it is easy to start simple and then grow as you require more complexity. Most administrators will not require each of the caching types.

When configuring caching, keep in mind the specific problems that you're trying to solve to avoid getting lost in the different implementation choices. Most users will benefit from at least setting up headers. If you are proxying or generating content, setting an HTTP cache may be helpful. Shared object caching is useful for specific tasks like storing SSL sessions or authentication details if you are using a backend provider. File caching can probably be limited to those with slow systems.



Reff :
https://www.digitalocean.com/community/tutorials/how-to-configure-apache-content-caching-on-ubuntu-14-04

Read More [...]

Posted in Ubuntu

Step by Step Install Ubuntu

First, you will need to download the current version of Ubuntu Server . A 64 bit version is recommended, but you can also download a 32bit version Here. Once downloaded you need to create a bootable CD/DVD or USB flash.

• How to burn a DVD on Windows
• How to create a bootable USB stick on Windows
• How to burn a DVD on Ubuntu
• How to create a bootable USB stick on Ubuntu

Once done, put the disc in your drive, or insert the USB flash, and reboot your machine. Make sure that you set your BIOS to boot either from the CD/DVD or USB flash depending on which installation method you choose.
Once booted, just follow the steps shown in the images below.

---

On the first screen you can choose you language of install.

---

Select Install Ubuntu Server.

---

Select the language used for the installation process.

---

Select your geographical location.

---

Configure your keyboard. I choose no to manually choose the keyboard.

---

I choose the English (US) for country origin.

---

I choose the English (US) keyboard layout.

---

At this point the system will detect the hardware to find a CD-ROM drive.

---

At this step, the installation will detect and load any additional component.

---

At this step, the installation will detect the network hardware.

---

Type in the host name you want to configure your server as.

---

Type in the user's name.

---

Type in a password for the username.

---

Choose whether to encrypt your home directory or not. I recommend selecting No I recommend selecting No because it makes it easier to do recovery if needed in case a corruption of data happens.

---

Configure clock, and time zone.

---

At this point, the installation is detecting disks, and other hardware.

---

At this stage is were you partition your disk(s). Select Guided as it is the default method.

---

Select the partition of your choice, is this installation its the SCSI1.

---

Select Yes to start writing the changes to disks, and configure LVM.

---

Answer Yes to actually write the changes to disk.

---

The installation is copying data at this stage.

---

At this stage the installation is configuring Apt Sources.List.

---

This is were you can configure a Proxy. I select Continue since I'm not using a proxy.

---

The installation is selecting the software being installed.

---

This is where you configure how to install updates. I choose the automatic method.

---

You can choose one these option at the time of install or later. if you select continue, you can manually choose what you want to install later.

---

Cleaning up is being done.

---

Starting the installation of Grub

---

Select Yes to install grub to the Master Boot Record

---

Finishing the installation. Select continue and remove the disk or USB.

You're done. Reboot your machine, and type in your username you created earlier, and your password to login. That is it, you are now logged in to your new Ubuntu server!

Read More [...]

Posted in Ubuntu

How To Protect SSH with fail2ban on Ubuntu 12.04

About Fail2Ban

Servers do not exist in isolation, and those virtual private servers with only the most basic SSH configuration can be vulnerable to brute force attacks. fail2ban provides a way to automatically protect virtual servers from malicious behavior. The program works by scanning through log files and reacting to offending actions such as repeated failed login attempts.

Step One—Install Fail2Ban

Use apt-get to install Fail2Ban

sudo apt-get install fail2ban

Step Two—Copy the Configuration File

The default fail2ban configuration file is location at /etc/fail2ban/jail.conf. The configuration work should not be done in that file, however, and we should instead make a local copy of it.

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

After the file is copied, you can make all of your changes within the new jail.local file. Many of possible services that may need protection are in the file already. Each is located in its own section, configured and turned off.

Step Three—Configure the Defaults in Jail.Local

Open up the the new fail2ban configuration file:

sudo nano /etc/fail2ban/jail.local

The first section of defaults covers the basic rules that fail2ban will follow. If you want to set up more nuanced protection on your virtual server, you can customize the details in each section.

You can see the default section below.

[DEFAULT]

# "ignoreip" can be an IP address, a CIDR mask or a DNS host
ignoreip = 127.0.0.1/8
bantime  = 600
maxretry = 3

# "backend" specifies the backend used to get files modification. Available
# options are "gamin", "polling" and "auto".
# yoh: For some reason Debian shipped python-gamin didn't work as expected
#      This issue left ToDo, so polling is default backend for now
backend = auto

#
# Destination email address used solely for the interpolations in
# jail.{conf,local} configuration files.
destemail = root@localhost

Write your personal IP address into the ignoreip line. You can separate each address with a space. IgnoreIP allows you white list certain IP addresses and make sure that they are not locked out. Including your address will guarantee that you do not accidentally ban yourself from your own server.

The next step is to decide on a bantime, the number of seconds that a host would be blocked from the VPS if they are found to be in violation of any of the rules. This is especially useful in the case of bots, that once banned, will simply move on to the next target. The default is set for 10 minutes—you may raise this to an hour (or higher) if you like.

Maxretry is the amount of incorrect login attempts that a host may have before they get banned for the length of the ban time.

You can leave the backend as auto.

Destemail is the email that alerts get sent to. If you have a mail server set up on your droplet, Fail2Ban can email you when it bans an IP address.

Additional Details—Actions

The Actions section is located below the defaults. The beginning looks like this:

#
# ACTIONS
#

# Default banning action (e.g. iptables, iptables-new,
# iptables-multiport, shorewall, etc) It is used to define
# action_* variables. Can be overridden globally or per
# section within jail.local file
banaction = iptables-multiport

# email action. Since 0.8.1 upstream fail2ban uses sendmail
# MTA for the mailing. Change mta configuration parameter to mail
# if you want to revert to conventional 'mail'.
mta = sendmail

# Default protocol
protocol = tcp
[...]

Banaction describes the steps that fail2ban will take to ban a matching IP address. This is a shorter version of the file extension where the config if is located. The default ban action, "iptables-multiport", can be found at /etc/fail2ban/action.d/iptables-multiport.conf

MTA refers to email program that fail2ban will use to send emails to call attention to a malicious IP.

You can change the protocol from TCP to UDP in this line as well, depending on which one you want fail2ban to monitor.

Step Four (Optional)—Configure the ssh-iptables Section in Jail.Local

The SSH details section is just a little further down in the config, and it is already set up and turned on. Although you should not be required to make to make any changes within this section, you can find the details about each line below.

[ssh]

enabled  = true
port     = ssh
filter   = sshd
logpath  = /var/log/auth.log
maxretry = 6

Enabled simply refers to the fact that SSH protection is on. You can turn it off with the word "false".

The port designates the port that fail2ban monitors. If you have set up your virtual private server on a non-standard port, change the port to match the one you are using:

 eg. port=30000

The filter, set by default to sshd, refers to the config file containing the rules that fail2ban uses to find matches. sshd refers to the /etc/fail2ban/filter.d/sshd.conf.

log path refers to the log location that fail2ban will track.

The max retry line within the SSH section has the same definition as the default option. However, if you have enabled multiple services and want to have specific values for each one, you can set the new max retry amount for SSH here.

Step Five—Restart Fail2Ban

After making any changes to the fail2ban config, always be sure to restart Fail2Ban:

sudo service fail2ban restart

You can see the rules that fail2ban puts in effect within the IP table:

sudo iptables -L

Reff :

https://www.digitalocean.com/community/tutorials/how-to-protect-ssh-with-fail2ban-on-ubuntu-12-04

Read More [...]

Posted in Ubuntu

Reset Your Forgotten Ubuntu Password root

If you’ve ever forgotten your password, you aren’t alone… it’s probably one of the most common tech support problems I’ve encountered over the years. Luckily if you are using Ubuntu they made it incredibly easy to reset your password.

 

All it takes is adjusting the boot parameters slightly and typing a command or two, but we’ll walk you through it.

Reset Your Ubuntu Password

Reboot your computer, and then as soon as you see the GRUB Loading screen, make sure to hit the ESC key so that you can get to the menu.

Root Shell – Easy Method

If you have the option, you can choose the “recovery mode” item on the menu, usually found right below your default kernel option.

Then choose “Drop to root shell prompt” from this menu.

This should give you a root shell prompt.

Alternate Root Shell Method 

If you don’t have the recovery mode option, this is the alternate way to manually edit the grub options to allow for a root shell.

First you’ll want to make sure to choose the regular boot kernel that you use (typically just the default one), and then use the “e” key to choose to edit that boot option.

 

Now just hit the down arrow key over to the “kernel” option, and then use the “e” key to switch to edit mode for the kernel option.

You’ll first be presented with a screen that looks very similar to this one:

You’ll want to remove the “ro quiet splash” part with the backspace key, and then add this onto the end:

rw init=/bin/bash

 

Once you hit enter after adjusting the kernel line, you’ll need to use the B key to choose to boot with that option.

 

At this point the system should boot up very quickly to a command prompt.

Changing the Actual Password

You can use the following command to reset your password:

passwd <username>

For example my username being geek I used this command:

passwd geek

After changing your password, use the following commands to reboot your system. (The sync command makes sure to write out data to the disk before rebooting)

sync
reboot –f

I found that the –f parameter was necessary to get the reboot command to work for some reason. You could always hardware reset instead, but make sure to use the sync command first.

And now you should be able to login without any issues.

Reff : http://www.howtogeek.com/howto/linux/reset-your-forgotten-ubuntu-password-in-2-minutes-or-less/

Read More [...]

Posted in Ubuntu

Installing CACTI and Templates on Debian Lenny/Squeeze

Installing CACTI and Templates on Debian Lenny/Squeeze

In the area of monitoring systems using GPL software Nagios is a big player but the organization of
displaying the history graphs is not so ideal.
Zabbix is wonderful as well, it produces also good graphs but it creates them dynamically, resulting
in a long wait time when we need to display a large amount of them at once. To achieve this Cacti
is one of the best GPL History-Graphs-Only monitoring program out there.
Note:
I have put together here a step-by-step HOWTO to help setting-up some of the most important items to be monitored in a short time. Although the theme says to use ‘Lenny’ the same should apply to Squeeze with very minor modification if any.
The difficulty with the version of Cacti provided with Debian Lenny/Squeeze is that it’s outdated.
I tried to install the latest version of Cacti and somehow got into some difficulties which
I felt I didn’t want to bother solving. I knew there was a faster solution to this and didn’t have the time
to do troubleshooting.
The fast solution is to install the regular (and outdated) Debian version of Cacti
and then replace its components with the latest version(0.8.7.g).
For later versions, simply replace the version number where applied. So here is how I did it:

Install the Debian Lenny/Squeeze version

apt-get install cacti
Note: Please make sure you complete the whole installation especially concerning the creation of the Mysql Database.

Download the current Cacti version

(at the time of this article the version 0.8.7g was the latest one.)
wget http://www.cacti.net/downloads/cacti-0.8.7g.tar.gz

Put the Debian Cacti files aside to leave space to the new one.

mv /usr/share/cacti /usr/share/cacti_Debian

Prepare space for the new version

mkdir -p /usr/share/cacti-0.8.7g

Install the new version with the same directory structure as with the Debian version

tar fvxz cacti-0.8.7g.tar.gz -C /usr/share/cacti-0.8.7g/
mv /usr/share/cacti-0.8.7g/cacti-0.8.7g /usr/share/cacti-0.8.7g/site
ln -s /usr/share/cacti-0.8.7g /usr/share/cacti
mv /usr/share/cacti-0.8.7g/site/rra /usr/share/cacti-0.8.7g/site/rra_orig
ln -s /var/lib/cacti/rra /usr/share/cacti-0.8.7g/site/rra
ln -s /usr/share/cacti-0.8.7g/site/resource /usr/share/cacti-0.8.7g/resource


Get a copy of the config.php from Debian version

This php file refers to the configuration file /etc/cacti/debian.php as installed by the original Debian version.
mv /usr/share/cacti/site/include/config.php /usr/share/cacti/site/include/config.php.orig
cp /usr/share/cacti_Debian/site/include/config.php /usr/share/cacti/site/include/config.php


Restart your web server

/etc/init.d/apache2 restart

Upgrade to the new cacti

http://myserver.com/cacti
and simply follow the on screen instructions to upgrade to the new version.
NOTES:
1) Debian Lenny/Squeeze uses the SNMP protocol Ver. 2.0 therefore make sure you do
configure the correct version in the cati web interface.
2) For better compatibility, in the /etc/snmp/snmpd.conf I used only the following configuration:
com2sec readonly default public
group MyROSystem v1 paranoid
group MyROSystem v2c paranoid
group MyROSystem usm paranoid
group MyROGroup v1 readonly
group MyROGroup v2c readonly
group MyROGroup usm readonly
group MyRWGroup v1 readwrite
group MyRWGroup v2c readwrite
group MyRWGroup usm readwrite
view all included .1 80
view system included .iso.org.dod.internet.mgmt.mib-2.system
access MyROSystem "" any noauth exact system none none
access MyROGroup "" any noauth exact all none none
access MyRWGroup "" any noauth exact all all none
syslocation Unknown (configure /etc/snmp/snmpd.local.conf)
syscontact Root (configure /etc/snmp/snmpd.local.conf)
Restart SNMPD Daemon
/etc/init.d/snmpd restart
Testing the SNMP configuration on the monitored system
snmpwalk -Os -c public -v 1 localhost system
3) I prefer to use the UDP Ping instead of the SNMP Ping for finding out if the server is available. This way if the SNMP service dies in the monitored host I get the proper information regarding if the monitored host died or not.
4) In our case, since we are going to get all sorts in information from the system where the user snmp is not allowed, we need to run the snmpd daemon as ‘root’ instead as ‘snmp’. To do this do the following changes:
vim /etc/default/snmpd
snmpd options (use syslog, close stdin/out/err).
SNMPDOPTS='-Lsd -Lf /var/log/snmpd.log -u root -I -smux -p /var/run/snmpd.pid 192.168.0.3,127.0.0.1'
Note: You also need to include the IP of the interface from where the monitoring server will be making its queries. In the above SNMPDOPTS settings, both interfaces having the IPs 192.168.0.3 and 127.0.0.1 will accept snmp queries. If any of the connected to the internet, make sure the /etc/hosts.allow/ & /etc/hosts.deny are correctly configured as follows:
Allowing the monitoring server acccess to snmpd daemon
In /etc/hosts.allow
snmpd: 90.78.145.72
Making sure all other hosts are denied access
In /etc/hosts.deny
snmpd: ALL

Installing some cacti templates

Apache statistics

In the monitored hosts
Make sure that in the client host you configure Apache for extended server-status and if needed configure the access control to accept the monitoring server where cacti runs.
In the monitoring server
– Download the apache template locally in desktop from
– Import the template in Cacti interface
wget http://public.itmatrix.eu/Cacti_templates/Apache/cacti_host_template_webserver_apache.xml
– Login as root:
cd /usr/share/cacti/site/scripts
wget http://public.itmatrix.eu/Cacti_templates/Apache/ws_apachestats.pl


Postfix queues

In the monitored host:
– download the postfix queues script in to /usr/local/bin/
cd /usr/local/bin/
wget http://public.itmatrix.eu/Cacti_templates/Postfix_queues/getmailq.sh
chmod 755 /usr/local/bin/getmailq.sh
Add the necessary SNMP configuration and restart
echo 'extend .1.3.6.1.4.1.2021.53.101.0 mailq /usr/local/bin/getmailq.sh' >> /etc/snmp/snmpd.conf
/etc/init.d/snmpd restart
Testing the script in the monitored host:
/usr/bin/snmpwalk -Oav -v2c -c public 127.0.0.1 .1.3.6.1.4.1.2021.53.101.0.4 | cut -d'"' -f2
You should get an output similar to this:
incoming:0 active:0 deferred:6 hold:0
In the monitoring server:
Log into the monitoring server and get the data query script
cd /usr/share/cacti/site/scripts
wget http://public.itmatrix.eu/Cacti_templates/Postfix_queues/postfixqueues.sh
chmod 755 /usr/share/cacti/site/scripts/postfixqueues.sh
Download and Import the template in cacti interface
http://public.itmatrix.eu/Cacti_templates/Postfix_queues/cacti_graph_template_postfix_queues.xml

Postfix statistics

In the monitored host
Download the postfix stats script in to /usr/local/bin/
cd /usr/local/bin/
wget http://public.itmatrix.eu/Cacti_templates/Postfix_stats/fetch_mail_statistics.pl
chmod 755 /usr/local/bin/fetch_mail_statistics.pl
Add the necessary SNMP configuration and restart
echo 'pass .1.3.6.1.4.1.2021.252 /usr/local/bin/fetch_mail_statistics.pl /var/log/mail.log /var/log/mailstats.db .1.3.6.1.4.1.2021.252' >> /etc/snmp/snmpd.conf
/etc/init.d/snmpd restart
Testing the script on the monitored host:
/usr/bin/snmpwalk -Oav -v2c -c public 127.0.0.1 .1.3.6.1.4.1.2021.252
You should get an output similar to this:
Output Description
============= ====================================
INTEGER: 25 Received
INTEGER: 1 Sent
INTEGER: 58 Deferred (Saved and to be resent later)
INTEGER: 0 Bounced (Bounced back)
INTEGER: 293 Rejected (Total rejected mails)
INTEGER: 15 Clean
INTEGER: 188 Pspam (Possible SPAM GreyListed)
INTEGER: 287 Spam (RBL and spammassassin Blocked)
INTEGER: 0 Infected (Blocked Virus)
INTEGER: 1 Bad Header
INTEGER: 1 Banned (Banned content)
In the Monitoring server (Via cacti web interface)
Download and Import the host template in cacti interface
http://public.itmatrix.eu/Cacti_templates/Postfix_stats/cacti_host_template_postfix_mailserver.xml

DISK IO statistics

In the monitored host:
Download the DISK IO stats script into /usr/local/bin/
cd /usr/local/bin/
wget http://public.itmatrix.eu/Cacti_templates/Disk_IO/snmpdiskio
chmod 755 /usr/local/bin/snmpdiskio

Add entries to /etc/snmp/snmpd.conf
echo 'extend .1.3.6.1.4.1.2021.54 hdNum /usr/local/bin/snmpdiskio hdNum' >> /etc/snmp/snmpd.conf
echo 'extend .1.3.6.1.4.1.2021.55 hdIndex /usr/local/bin/snmpdiskio hdIndex' >> /etc/snmp/snmpd.conf
echo 'extend .1.3.6.1.4.1.2021.56 hdDescr /usr/local/bin/snmpdiskio hdDescr' >> /etc/snmp/snmpd.conf
echo 'extend .1.3.6.1.4.1.2021.57 hdInBlocks /usr/local/bin/snmpdiskio hdInBlocks' >> /etc/snmp/snmpd.conf
echo 'extend .1.3.6.1.4.1.2021.58 hdOutBlocks /usr/local/bin/snmpdiskio hdOutBlocks' >> /etc/snmp/snmpd.conf
Restart snmpd service
/etc/init.d/snmpd restart
Testing the data query on the monitored host:
/usr/bin/snmpwalk -Oav -v2c -c public 127.0.0.1 .1.3.6.1.4.1.2021.54.4
/usr/bin/snmpwalk -Oav -v2c -c public 127.0.0.1 .1.3.6.1.4.1.2021.55.4
/usr/bin/snmpwalk -Oav -v2c -c public 127.0.0.1 .1.3.6.1.4.1.2021.56.4
/usr/bin/snmpwalk -Oav -v2c -c public 127.0.0.1 .1.3.6.1.4.1.2021.57.4
/usr/bin/snmpwalk -Oav -v2c -c public 127.0.0.1 .1.3.6.1.4.1.2021.58.4
In the Monitoring server:
Install the needed xml file
mkdir -p /usr/share/cacti/extras
cd /usr/share/cacti/site/resource/
wget http://public.itmatrix.eu/Cacti_templates/Disk_IO/partition.xml
ln -s /usr/share/cacti/site/resource/snmp_queries/partition.xml /usr/share/cacti/extras/partition.xml

Download on local desktop and import the host template using the Cacti web interface
http://public.itmatrix.eu/Cacti_templates/Disk_IO/cacti_data_query_snmp_disk_statistics.xml
http://public.itmatrix.eu/Cacti_templates/Disk_IO/cacti_graph_template_disk_io_bytessec.xml

Advanced MySQL statistics

In the monitored host:
– Create a special user for monitoring in MySQL server with the same username and password for all monitored hosts
– This user must have the SUPER_Privilege only
– Use ‘PASSWORD’ type and NOT MD5 or something else
Make sure the mysql server binds to ALL network interfaces:
Edit /etc/mysql/my.cnf
[mysqld]
........
#bind-address = 127.0.0.1
bind-address = 0.0.0.0
Protect the mysql server from Internet access using a firewall or the /etc/hosts.allow & /etc/hosts.deny as follows:
————————————-
Allowing the monitoring server acccess to snmpd daemon
In /etc/hosts.allow
mysqld: 127.0.0.1 192.168.100.0/255.255.255.0
Making sure all other hosts are denied access
In /etc/hosts.deny
mysqld: ALL
In the Monitoring server:
Download and install the php script
cd /usr/share/cacti/site/scripts/
wget http://public.itmatrix.eu/Cacti_templates/Improved_Mysql/ss_get_mysql_stats.php
ln -s /usr/share/cacti/site/scripts/ss_get_mysql_stats.php /usr/share/cacti/extras/

Edit the name and password in /usr/share/cacti/site/scripts/ss_get_mysql_stats.php
eg.
$mysql_user = 'monitor';
$mysql_pass = 'monitor';

Download on desktop and Import the host template using the cacti web interface
http://public.itmatrix.eu/Cacti_templates/Improved_Mysql/cacti_host_template_x_db_server_ht_0.8.6j.xml

TeMySQL statistics

In the monitored host:
– Create a special user for monitoring in MySQL server with the same username and password for all monitored hosts
– This user must have the SUPER_Privilege only
– Use ‘PASSWORD’ type and nothing else
Make sure the mysql server binds to ALL network interfaces:
Edit /etc/mysql/my.cnf
[mysqld]
........
#bind-address = 127.0.0.1
bind-address = 0.0.0.0

Protect the mysql server from Internet access using a firewall or the /etc/hosts.allow & /etc/hosts.deny as follows:
Allowing the monitoring server acccess to snmpd daemon
In /etc/hosts.allow
mysqld: 127.0.0.1 192.168.100.0/255.255.255.0
Making sure all other hosts are denied access
In /etc/hosts.deny
mysqld: ALL
In the Monitoring server:
Download and install the php scripts
cd /usr/share/cacti/site/scripts/
wget http://public.itmatrix.eu/Cacti_templates/teMySQLcacti/mysql_stats.php
ln -s /usr/share/cacti/site/scripts/mysql_stats.php /usr/share/cacti/extras/

Download on desktop and Import the host template in cacti interface
http://public.itmatrix.eu/Cacti_templates/teMySQLcacti/cacti_host_template_temysql_host-step300-heartbeat600.xml
During the creation of teMysql graphs in each device, enter the name and password of the mysql user configured in the monitored host for this purpose.
eg.
Name: monitor
PW: monitor


Network statistics

In the monitoring server:
Log into the monitoring server and get the data query script
cd /usr/share/cacti/site/scripts
wget http://public.itmatrix.eu/Cacti_templates/Netstat/lvm_netstat_tcp.pl
chmod 755 /usr/share/cacti/site/scripts/lvm_netstat_tcp.pl
ln -s /usr/share/cacti/site/scripts/lvm_netstat_tcp.pl /usr/share/cacti/extras/

Download and Import the template using the cacti web interface
http://public.itmatrix.eu/Cacti_templates/Netstat/cacti_graph_template_snmp_get_tcp_connection_status.xml

Advanced Ping

In the monitoring server:
Log into the monitoring server and get the data query script:
cd /usr/share/cacti/site/scripts
wget http://public.itmatrix.eu/Cacti_templates/Advanced_ping/ss_fping.php
ln -s /usr/share/cacti/site/scripts/ss_fping.php /usr/share/cacti/extras/

Download and Import the template in cacti interface
http://public.itmatrix.eu/Cacti_templates/Netstat/cacti_graph_template_ping_advanced_ping_v1_3.xml

SNMP Disk Usage

In the monitoring server:
Log into the monitoring server and get the query xml file
cd /usr/share/cacti/site/resource/snmp_queries/
wget http://public.itmatrix.eu/Cacti_templates/hrStorageTable/hrStorageTable.xml
ln -s /usr/share/cacti/site/resource/snmp_queries/hrStorageTable.xml /usr/share/cacti/extras/

Download and Import the template in cacti interface
http://public.itmatrix.eu/Cacti_templates/hrStorageTable/cacti087d_data_query_snmp_-_hrstoragetable.xml

HTTP Response Time

In the monitoring server:
Log into the monitoring server and get the script file
cd /usr/share/cacti/site/scripts
wget http://public.itmatrix.eu/Cacti_templates/HTTP_response_time/http_response_time.pl
ln -s /usr/share/cacti/site/scripts/http_response_time.pl /usr/share/cacti/extras/

Download and Import the template using cacti web interface
http://public.itmatrix.eu/Cacti_templates/HTTP_response_time/cacti_data_template_http_reponse_time.xml
http://public.itmatrix.eu/Cacti_templates/HTTP_response_time/cacti_graph_template_http_response_time.xml

NOTE: If by any chance you get no graphs created after a good 10 minutes then look at this solution:
Click on ‘Graph Management’, Click on one of the ‘HTTP Response Time ‘ graphs links and if you get no graph and the ‘RRDTool Says:’
ERROR: the RRD does not contain an RRA matching the chosen CF
Then the following might be the reason:
You are using (the Graph Template includes) the LAST consolidation function.
But your RRA definitions do not include this CF.
Edit RRAs when clicking onto “Data Sources”, then select “RRAs”
and edit all of them to add “LAST” CF
Then, you’ll have to re-create the failing rrd file
By deleting the graph from Graph management and under each device create the graph again.

PostGres Statistics

PostGres Data Base Cacti templates:
Ref: http://forums.cacti.net/viewtopic.php?f=12&t=23300
Installation instruction:
1. Make sure you have enabled Statistic Collector in Postgres (postgresql.conf)
Till postgres 8.2
stats_start_collector = true
stats_command_string = true
stats_block_level = true
stats_row_level = true
stats_reset_on_server_start = false

From postgres 8.3
track_activities = on
track_counts = on
update_process_title = on

Also in this configuration file:
Make sure PostGresql server listens to the interfaces that will be used for connections:
Here you give the IP of the server’s local interfaces.
eg.
listen_addresses = '127.0.0.1,192.168.100.117'
2. Edit the access rights file /etc/postgresql/8.3/main/pg_hba.conf
Then make sure the clients will be allowed in via the same interface Postgres is listening on above:
Add the access rights line:
host all all 192.168.100.0/24 md5
3. Create new User in PostgreSQL ,for example ‘monitor’
(you don’t have to grant any special roles to this users AFAIK, any user has access to Statistic Tables).
su - postgres
createuser --pwprompt --encrypted --no-adduser --no-createdb monitor
– Enter password for new role: monitor
– Enter it again: monitor
– Shall the new role be allowed to create more new roles? (y/n) n
4. Try to connect remotely under this user to postgres Database:
psql -h YOUR_POSTGRES_HOST -U monitor -W postgres
and try execute following SQL:
select * from pg_stat_all_tables;
If everything work proceed with installation otherwise fix your problem till you get results.
5.In monitoring server:
Commands:
cd /usr/share/cacti/site/scripts
wget http://public.itmatrix.eu/Cacti_templates/PostGres/postgresql_database_stats/pgsql_stats.php
cd /usr/share/cacti/site/resource/script_queries
wget http://public.itmatrix.eu/Cacti_templates/PostGres/postgresql_database_stats/postgres_dbstat.xml

Description of templates.
pgsql_stats.php - the Script, put this in /scripts/
postgres_dbstat.xml - the Definition, put this in /resource/script_queries
cacti_data_query_postgresql_database_stats.xml - Data Query + Graphs - import into Cacti
6. Install the debian package ‘php-pgsql’
apt-get install php5-pgsql
/etc/init.d/apache2 restart
7. Download and Import the template using cacti web interface
http://public.itmatrix.eu/Cacti_templates/PostGres/postgresql_database_stats/cacti_data_query_postgresql_database_stats.xml
8. Edit pgsql_stats.php line 173 and 174 and put username and password you have created at point 1.
(I don’t know how to pass username and password parameters to Data Queries. If you know how,
let me know so I can remote username and password from script)
9. Edit your Host Template or Device and add PostgreSQL Database Stats in Data Queries.
10. Create your Graphs :)

My Processes

Note: this self made cacti monitoring template does the monitoring of the number of processes per specific services as follows:
OUTPUT example:
allprocs:226 apache2:12 mysqld:19 mysqld2:0 bash:1 gpg:0 pop3:3
Its called : MyProcessesMGT2 in the monioring list
In the monitoring server:
– get the script file
cd /usr/share/cacti/site/scripts
wget http://public.itmatrix.eu/Cacti_templates/My_Processes/my_processes.sh
chmod 755 /usr/share/cacti/site/scripts/my_processes.sh

Make sure the user www-data on the monitoring server can run via ssh without password the script /usr/share/cacti/site/scripts/my_processes.sh
Commands in monitoring server:
mkdir /home/www-data
usermod -d /home/www-data -s /bin/bash www-data
chown www-data. /home/www-data
su - www-data
ssh-keygen -t rsa
(just press Enter key at every question)

In each monitored hosts:
-Modify the user www-data in each monitored hosts, create a home directory for it
-Commands in monitored hosts:
mkdir /home/www-data
usermod -d /home/www-data -s /bin/bash
chown www-data. /home/www-data
passwd www-data
(Give 2 times the password ‘www-data’ (no worry it will be deleted right after we are finished)
When all monitored hosts are modified as above then go back to the monitoring server
Commands in monitoring server:
su - www-data (only if not already logged-in as www-data)
ssh-copy-id clientHost_1_Name (Give the 'www-data' password)
ssh-copy-id clientHost_2_Name "" "" ""
ssh-copy-id clientHost_3_Name "" "" ""
........

Try the connection
IMPORTANT: use the same hostname exactly as it is given as hostname in cacti for every monitored host here as clientHost_1_Name. If you get a question like:
Are you sure you want to continue connecting (yes/no)?
Answer ‘yes’.
ssh clientHost_1_Name (if all ok then CTRL-D to exit the session on this remote host)
ssh clientHost_2_Name (if all ok then CTRL-D to exit the session on this remote host)
ssh clientHost_3_Name (if all ok then CTRL-D to exit the session on this remote host)
.......

Disable all password of www-data in all monitored hosts
Make sure you are back loggged in as ‘root’
Commands in all monitored hosts:
ssh clientHost_1_Name sed -i 's/www-data:/www-data:*/' /etc/shadow
ssh clientHost_2_Name sed -i 's/www-data:/www-data:*/' /etc/shadow
ssh clientHost_3_Name sed -i 's/www-data:/www-data:*/' /etc/shadow
.........

Download and Import the template in cacti interface:
http://public.itmatrix.eu/Cacti_templates/My_Processes/cacti_data_template_myprocessesmdt2.xml
http://public.itmatrix.eu/Cacti_templates/My_Processes/cacti_graph_template_myprocessesmgt2.xml





nb: link from : http://tipstricks.itmatrix.eu/?p=721

Read More [...]

Posted in Cacti

Limit Extention dengan Layer 7 Mikrotik

Untuk melimit melalui layer 7, maka script yang harus di tambahkan yaitu :

/ip firewall layer7-protocol
add comment="" name=High regexp="^.*get.+\\.(exe|rar|iso|zip|7zip|0[0-9][1-9]|flv|mkv|avi|mp4|3gp|rmvb|mp3|img|dat|mov).*\$"
add comment="" name=Mid regexp="^.*get.+\\.(zip|rar|7z).*\$"
add comment="" name=End regexp="^.*get.+\\.(pdf|doc|docx|xlsx|xls|rtf|ppt|ppt).*\$"
add comment="" name=Video regexp="http/(0\\.9|1\\.0|1\\.1)[\\x09-\\x0d ][1-5][0-9][0-9][\\x09-\\x0d -~]*(content-type: video)"

Menggunakan Regular Expersion silahkan googling untuk penjelasanya.
High = File yg biasa berukuran besar dan paling sering didownload
Mid = File bertype zip rar 7z berukuran hanya sampai 10 MB jika lebih akan dilimit
End = File yg tidak ingin di limit
Video = Stream Video

Kedua tandai mangle


/ip firewall mangle
add action=mark-packet chain=forward comment="High Eks" disabled=no dst-address-list=!Bypass layer7-protocol=High new-packet-mark=High-Ext passthrough=no protocol=tcp
add action=mark-packet chain=forward comment="Mid Eks" connection-bytes=10485760-4294967295 disabled=no layer7-protocol=Mid new-packet-mark=Mid-Ext passthrough=no protocol=tcp
add action=mark-packet chain=forward comment="Low Eks" disabled=no layer7-protocol=End new-packet-mark=End-Ext passthrough=no protocol=tcp
add action=mark-packet chain=forward comment=Video disabled=no layer7-protocol=Video new-packet-mark=Video passthrough=no protocol=tcp src-address-list=!Bypass

Unttuk selanjutnya... di queue nya setting sendiri ya.. ^_^

Read More [...]

Posted in Mikrotik