Nginx Proxy with Virtual Hosts / Server Blocks

Background:

I have an amazing home internet connection, with 1G up and down. This is perfect for hosting my own servers at home. As a result I run an ESXi host with many VMs. My standard process is to spin up a new VM for each service that I run, giving each service effectively a disposable nature. Providing I have enough memory in the server this approach is perfect. A simple ubuntu VM with 1G of RAM is often more than enough to run any of my single services.

Problem:

While the ability to run all these VMs and host them on a 1G internet connection is perfect, I only have a single IP to play with. My ISP does not give me any more IPv4 addresses and does not support IPv6 at all!

Some of these services are external and web based (this blog, for example). Using a nice straightforward NAT rule we can port-forward port 80 external to my internal web servers.

Now this approach works nicely when all services are running on the same server under the same process. I have some servers running nodejs, some apache and some nginx. Some are appliances, some are just general websites etc etc, what a mess! Some are even on non-standard ports! How on earth do we access all of these without re-mapping ports to different servers and having to specify the ports manually.

I want to be able to connect to all of these services externally, on port 80, by simply specifying a different hostname. For example, GHOST_URL/ goes to my blog, http://file.dchidell.com to go to a fileserver, http://testweb.dchidell.com to a testing webserver etc. These hostnames would all point towards the same IP address.

Solution:

Overview

Nginx proxy! Nginx has an amazing capability to proxy HTTP requests to another servers. The configuration is actually quite simple and scales reasonably well for the servers in question.

Slightly crude diagram but it demonstrates the point!

Implementation

Disclaimer: I'm no expert in nginx, this configuration works fine but as to its efficiency I cannot comment. There may be better ways of achieving this with less configuration or more efficiently within nginx.

Installing nginx is the easy part, I use ubuntu server for my installations so apt-get install nginx is about as easy as it gets.

Similar to apache nginx uses sites-available and sites-enabled folders within the /etc/nginx directory. Inside /etc/nginx/sites-available a default site should exist, this can be removed but it's probably better just to remove the symbolic link from sites-enabled: rm /etc/nginx/sites-enabled/default

Now for the useful part, I create a separate file for each server I'm talking to, I find this personally easier as all I have to do is copy a file and change a few pieces of information. The sample file is below:

upstream blog 
{
        server 10.8.77.26:8880;
}


server 
{
        listen 80;


        server_name blog blog.dchidell.com;

        location / 
        {
                proxy_pass http://blog;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
        }
}

The upstream section contains the server I'm pushing requests to. If you exclude the port number it will be the default of 80.

The server section describes the bulk of configuration, I want to connect on port 80. The server name is the important part, this is the hostname which you're connecting from externally. I've added blog and blog.dchidell.com because internally it's quite useful to just be able to go to http://blog and it works, without typing the whole FQDN.

The rest of the information corresponds to various information which usually gets rewritten but is quite useful to preserve so the webserver can actually see the real IP which it's getting accessed from etc.

So the lines which would require changing are:

• upstream blog - Replace blog with some server definition.
  
• server 10.8.77.26:8880; - Replace The IP / port with the actual server where your content is hosted.
  
• listen 80; - If you want to change the port your entire lot of web services is running, change this.
  
• server_name blog blog.dchidell.com; - Replace with the URL you want the above service to be accessed from.
  
• proxy_pass http://blog; - Replace blog with whatever was specified in the upstream blog section.

Now after you've changed everything in these files to reflect what you want inside /etc/nginx/sites-available these files must be linked to sites-enabled so you can actually use them. Some apache users may recall a a2ensite and a2dissite which you can use to manage these virtual hosts (or server blocks as nginx refers to them as). However nginx requires a little more manual approach, best for this is to use symbolic links. This can be achieved using the following command:
ln -s /etc/nginx/sites-available/blog /etc/nginx/sites-enabled/blog

If you're feeling cautious you can test the configuration using service nginx configtest however I'm a little more throw caution to the wind and go for the full on service nginx restart!

Everything should now be working! Adding new servers / virtual hosts is easy enough, just create a new file in the sites-available directory, add the symbolic link and restart nginx. Easy enough!

EDIT: I've recently had some trouble with file servers sitting behind the reverse proxy. This is due to nginx buffering the connection and only buffering 1G of data. I circumvented this by adding the following to the server block: proxy_buffering off;