Apache » OpenAlfa Blog

How to limit the crawl rate of bots in a website

Apache, Server Administration No Responses »

Jul 172014

Increasing the volume of visits is one of the basic objectives of most websites. And in most cases, the greater number of visits comes from the main internet search engines, such as Google and Bing.

Indeed, for the pages in our site to appear in the search result pages of these search engines, they must have previously been added to their indexes.

To index a site, search engines run specialized applications commonly referred to as “bots”, “crawlers” or “spiders”. These bots navigate the content of web sites, reading the content of the pages they find.

In principle, being actively crawled by different search engines it is a good signal for a site. But it may happen that the number of accesses done by crawlers becomes excessive, affecting the performance of the server, making it appear as slow or unresponsive to users. In this post we will review different possibilities to limit the crawl rate of the main internet search engines. Continue reading »

How to configure a proxy server in Apache

Apache, Server Administration No Responses »

Jun 122014

In the first place, a distinction needs to be made between forward proxies and reverse proxies.

How to configure HTTP redirects 301, 302, 403, 404 and 410 using RewriteRule in Apache

Apache No Responses »

May 142014

Sometimes, the webmaster of a website may consider that the structure of the content of the site should be modified in some way. As a result,the URLs used to access some pages might be affected. But in other pages there may be links (either external links from other sites, or internal links) to those pages, using the old URL. Besides, search engines such as Google might already have indexed the content of the page, also using the old URL.

If the URL is modified without taking into account this fact, all those link would become broken links, and the ranking of the linked pages would most likely be affected as well.

To prevent this undesirable effect, a RewriteRule directive can be put in the config file of the Apache web server, to redirect the old URLs to the new ones, as explained in this post.

Coding a web page cache in PHP

Apache, PHP No Responses »

Apr 042014

Many web sites offer dynamic content that results from the execution of a script, in many cases written in PHP. Every time the web server receives a request, the script that generates the HTML code returned to the client runs,often performing several database queries, or any other type of processing that might be expensive in time and resource usage, resulting in a slow response time and poor user experience.

This post explains the implementation of a cache mechanism that may alleviate this problem: The HTML code resulting from the execution of the script is saved to disk, and every time a new request is received for the same page, this pre-processed content is served directly from the disk cache, thus avoiding the need to execute again the whole process.

How to configure the https protocol in Apache

Apache No Responses »

Mar 072013

A web site for e-commerce, or any other kind of service that receives confidential information from users, must protect the trasmission of that information. But the HTTP protocol is not secure for this type of use.

The most common way to improve the security of a site is to configure the web server to establish SSL connections with the browsers. HTTP data interchanged between clients and server through this kind of connection is sent encrypted. the combination HTTP+SSL is known as HTTPS (Secure HTTP).

This post details the steps to follow to configure the https protocol on an apache web server on a Linux system.

How to monitor the status of services in a web server

Apache, MySQL, Server Administration No Responses »

Jan 042013

A typical internet web server, either a WordPress blog, or a site running Joomla, Drupal or other kind of Content Management System, requires at least two services that must be continuously running; an apache web server, and a MySQL DBMS.

But even the in most stable server, it may happen that one of these services meets an unexpected condition that forces it to stop working.

For this reason, it is very desirable to have a “service monitor” script to check periodically the status of these critical services, and perform the required actions when it finds that one of them is not running, such as sending an alert by email, and trying to start it again.

In this post a simple script is presented that monitors the status of the apache and mysql services on a linux server. The sample script can easily be extended to monitor other critical services as well.

How to configure Apache mpm worker

Apache No Responses »

Dec 042012

An Apache web server running in worker multiprocessing mode forks a set of children processes to answer incoming requests from users. Each of these children processes, in turn, spawns several threads to distribute the processing load. The number of processes and threads to run can be configured to optimize the usage of the available resources.

How to increase the performance of a WordPress site with Apache worker mpm

Apache, WordPress No Responses »

Nov 272012

An Apache web server can employ different multiprocessing strategies to benefit from the CPU and RAM resources available at the system where it runs. Among them, the most common ones are the “prefork” and “worker” multiprocessing modules (mpm).

Multithreading in PHP

Apache, PHP 2 Responses »

Nov 262012

For a long time, PHP has been lacking the functionality to write programs able to run several threads in parallel.

Currently, it is possible to install a PHP interpreter with pthreads (POSIX threads) support. However, the standard PHP distributions do not include this build.

In order to obtain a multithread-enabled PHP interpreter it, is necessary to compile it from source, including in the configure step the option “–enable-maintainer-zts” (–enable-zts on Windows). This generates a “thread-safe” interpreter that can be run under an Apache server with multithreading (worker MPM), instead of a simple multiprocessing Apache server (pre-fork MPM). However, this type of installation is generally not recommended because, even if the PHP core is thread-safe, some of the extensions used might not be thread-safe.

In this post, we will explore the existing possibilities to implement multi-threading in PHP, and the considerations to take into account in doing so.

How to process the error logs in a web server

Apache, Server Administration No Responses »

Nov 042012

We have covered in previous posts some alternatives to process the access logs of a web server.

It is equally important to analyze the error log, to detect errors that could be affecting the service. This post presents a simple script to automate this analysis.

Older Entries