miércoles, mayo 02, 2007

The long and winding road to a web farm

It all started in the last part of 2005. I had been playing with the idea of setting up a full featured website, but had not been successful finding anyone to join. So I decided to start on my own.

The Zero Investment Promise (ZIP)
I didn't have the money or the will to spend it on this project, but I had the feeling I could make it by using only legal ways as long as I were ready to use my brain. That would also exclude hacking or stealing resources from others, or even copying unauthorized materials.

The initial equipment
All I got to start was a dynamic IP DSL line with just 1MB bandwidth, a 3COM wireless router and a low-end desktop computer running Windows XP.

Along the way, other software will be needed. All free. So our ZIP is secured.
  1. Apache
  2. PHP
  3. MySQL
  4. Joomla and Joomla extensions
  5. Windows Update client

The web server
The first step was to choose a web server. I did some testing with IE, but decided for Apache 2 because it seemed much more feature-rich and documented. I also heard it was faster, so I quickly set a basic installation on my home desktop. Once I was able to open a browser, typing localhost on the address bar and get the welcome Apache page, I knew that was a small step for a man, but a giant step for my ego.

Once the server was running, I had to find a way to allow people from outside my home network to call. I had no idea how to do it, but a nice German workmate accessed my router remotely and showed me the way by using a feature called virtual servers. Now I was able to route any incoming request to a given internal IP. That meant I had to give my desktop a fixed IP so the router would find it.

The Dynamic DNS
At this point of my journey, I had a functional server, but since I was getting a new IP from my ISP every few hours, I needed to take note of my newly assigned IP and use it to call my http server from outside.

Since you will probably choose to name your websites to reach them, like respectable people do, a single subscription is needed. Go to DynDNS and create an account to use their free DNS service for your dynamic IP addresses. This way, everytime your ISP wickedly changes your IP, a piece of software in your own network will update the DNS, so you will be able to use a name just like if you had rented a fixed IP plus a domain.

The piece of software I am referring to can be located either in
  • your own router (simpler, no need to download and install software) or
  • directly on your own host (more reliable, flexible, needed if you have more than one website).
Luckily, by using a feature called DDNS on my router, I was able to tell it to refresh my IP on one of the two suggested Dynamic DNS services. I chose DynDNS, and set up an account and a domain name without any problem.

Now, my server was working more or less like a commercial host, answering all incoming http calls through the router.

Anyway, I noticed my router-only solution was not as fast as anticipated updating changing IP's to my DNS, so I took the advice from DynDNS guys and set up in my host a client updater too to reinforce the service. Both systems can work redundantly. Why would you choose just one? I like redundancies, especially in women.

Besides, this proved to be a wise idea once I decided I needed to serve more than one website from the same IP. But I will tell you later about that one.

Shotting websites from home
Hey, it was nice to have my own web server. I know hosting and hiring a domain is becoming a cheap thing, but hiring a fixed IP is not in Spain. By using a dynamic DNS, I was keeping my promise on Zero Investments.

I placed my old personal website on my newly opened web server. It worked flawless.

The good old but little-known e-mail server
All websites needing to log users in need a way to send them an e-mail during the registering process. There are several free solutions you can use. I decided for a Windows-specific open source software, hMailServer.

The real problem there was that e-mail coming from servers using a dynamic IP are not much popular among commercial email services because they are usually pested with spam, so Hotmail and others will block mails coming for your home e-mail server as long as its IP is marked as being within a Dynamic range. There's nothing you can do about it.

Confronted with the idea of a home e-mail server only to use its POP3 or IMAP4 service, I decided to completely abandon it and use my old hotmail account to receive, and my ISP's SMTP service to send.

The Great Challenge: a CMS
I downloaded, installed and ran phpBB and 4Images packages without any problem, even when right with the first one I was told to install MySQL first. Some of them will create on their own the databases needed, other may ask you to do some typing in the command line to do it. But it's a straight-away process, so do not worry.

I liked what I saw. Yeah. But I was looking for an integrated solution so I could add more functionalities, like the portals you see around on the net. Again, my old pride.

I decided for Joomla. It seemed more updated and active than Mambo, and started by setting a testing portal with no extensions.

Looking for customers
I offered my local tennis club a customized website for free... They liked the idea, so for the coming months I tailored a template, and added a bunch of extensions. Calendars, forums, weather reports, multimedia galleries... just name it... Almost nothing worked, hmm, let's say out of the box... but a few postings in the right forums and here you are. A glittering website much more sophisticated than most portals around.

What is SEO?
Search Engine Optimization is an essential part of the fun, as long as you want your website to be read by others that are not your wife or your daughter.

AdSense, Analytics, Sitemaps and other Google candies
Now this is great. Joomla comes with a simple banner component, so you can set accounts and banners for selling impressions, but you still need to knock on your prospect sponsors' door for the money.

If you hate the job, you can still get some bucks by creating an AdSense account and adding any AdSense module to your Joomla site. Soon you may check how popular (or most likely totally unknown) your site is. They will also give you hints on the next steps to take to add a new dimension to your websites: profitability.

The server grows. A web farm flourishes.
This Apache feature is called name-based virtual servers, so you can serve several websites from the same unique IP machine. Isn't it lovely? It's quite easy to use, believe me. All you have to do is some editing in the httpd.conf file. Apache 2.2 proposes these extra configurations to be written in separate files, but you can still put everything together in httpd.conf, instead.

But remember that, since you are using a dynamic IP, once your Apache starts to serve several websites, you will have to:
  1. add a new host service for each additional website to your account at DynDNS
  2. add these new hosts to your DynDNS updater.
Securing the farm
You have already heard of many dangers surrounding a website, no matter how humble it may be. The fact is that you have to take all the usual cautions to shield your Apache server up from day one.

Regarding the individual websites, well... the ones using PHP as Joomla does always come with a user management module. So, dynamic pages will only be built and posted using the user rights. A problem appears with Static pages. These old but plenty of good memories all pages are ready to show to everyone, as long as they are requested. For them, you have to use the authentication built-in features of Apache. Not big deal. Just:
  1. Create the user/password list or database, using the htpasswd utility found in Apache folders, and place it where you may reach it later... just in case you didn't figure out. You can specify to encrypt your passwords using MD5 or SHA, so no one accessing it will make any profit. Not even account owners...
  2. Then you have to edit the httpd.conf file to make Apache to ask for a valid user/password every time someone requests a website within the protected directories. You can here decide whether you want Apache to encrypt the password as it is trasmitted from the user, so no one listening will make any profit, neither.
That's about it. You can instead encrypt everything using SSL Apache implementation feature, as long as you really need that level of privacy and you are ready to pay the extra workload it takes.