{"id":76010,"date":"2020-01-20T12:51:30","date_gmt":"2020-01-20T07:21:30","guid":{"rendered":"https:\/\/www.vskills.in\/certification\/tutorial\/?p=76010"},"modified":"2024-04-12T14:17:44","modified_gmt":"2024-04-12T08:47:44","slug":"scaling-2","status":"publish","type":"page","link":"https:\/\/www.vskills.in\/certification\/tutorial\/scaling-2\/","title":{"rendered":"Scaling"},"content":{"rendered":"

Now that you know how to get Django running on a single server, let\u2019s look at how you can scale out a Django installation. This section walks through how a site might scale from a single server to a large-scale cluster that could serve millions of hits an hour.<\/p>\n

It\u2019s important to note, however, that nearly every large site is large in different ways, so scaling is anything but a one-size-fits-all operation. The following coverage should suffice to show the general principle, and whenever possible we\u2019ll try to point out where different choices could be made. First off, we\u2019ll make a pretty big assumption and exclusively talk about scaling under Apache and mod_python. Though we know of a number of successful medium- to large-scale FastCGI deployments, we\u2019re much more familiar with Apache.<\/p>\n

Running on a Single Server<\/strong> – Most sites start out running on a single server, with an architecture that looks something like Figure 20-1.<\/p>\n

\"image029\"<\/p>\n

Figure 20-1: a single server Django setup.<\/p>\n

 <\/p>\n

This works just fine for small- to medium-sized sites, and it\u2019s relatively cheap \u2014 you can put together a single-server site designed for Django for well under $3,000. However, as traffic increases you\u2019ll quickly run into resource contention<\/em> between the different pieces of software. Database servers and Web servers love<\/em> to have the entire server to themselves, so when run on the same server they often end up \u201cfighting\u201d over the same resources (RAM, CPU) that they\u2019d prefer to monopolize. This is solved easily by moving the database server to a second machine.<\/p>\n

Separating Out the Database Server<\/strong><\/p>\n

As far as Django is concerned, the process of separating out the database server is extremely easy: you\u2019ll simply need to change the DATABASE_HOST setting to the IP or DNS name of your database server. It\u2019s probably a good idea to use the IP if at all possible, as relying on DNS for the connection between your Web server and database server isn\u2019t recommended. With a separate database server, <\/span>our architecture now looks like Figure 13-2.<\/span><\/p>\n

 <\/span><\/p>\n

Here we\u2019re starting to move into what\u2019s usually called n-tier architecture. Don\u2019t be scared by the buzzword \u2013 it just refers to the fact that different tiers of the Web stack get separated out onto different physical machines.<\/span><\/p>\n

 <\/span>At this point, if you anticipate ever needing to grow beyond a single database server, it\u2019s probably a good idea to start thinking about connection pooling and\/or database replication. Unfortunately, there\u2019s not nearly enough space to do those topics justice in this book, so you\u2019ll need to consult your database\u2019s documentation and\/or community for more information.<\/span><\/p>\n

 <\/span><\/p>\n

\"image031\"<\/strong><\/p>\n

Figure : Moving the database onto a dedicated server.<\/span><\/p>\n

Running a Separate Media Server<\/strong><\/p>\n

We still have a big problem left over from the single-server setup: the serving of media from the same box that handles dynamic content. Those two activities perform best under different circumstances, and by smashing them together on the same box you end up with neither performing particularly well. So the next step is to separate out the media \u2014 that is, anything not<\/em> generated by a Django view \u2014 onto a dedicated server.<\/p>\n

\"image033\"<\/p>\n

Figure 20-3: Separating out the media server.<\/p>\n

 <\/p>\n

Ideally, this media server should run a stripped-down Web server optimized for static media delivery. lighttpd and tux (http:\/\/www.djangoproject.com\/r\/tux\/) are both excellent choices here, but a heavily stripped down Apache could work, too. For sites heavy in static content (photos, videos, etc.), moving to a separate media server is doubly important and should likely be the first<\/em> step in scaling up.<\/p>\n

This step can be slightly tricky, however. Django\u2019s admin needs to be able to write uploaded media to the media server (the MEDIA_ROOT setting controls where this media is written). If media lives on another server, however, you\u2019ll need to arrange a way for that write to happen across the network.<\/p>\n

Implementing Load Balancing and Redundancy<\/strong><\/p>\n

At this point, we\u2019ve broken things down as much as possible. This three-server setup should handle a very large amount of traffic \u2014 we served around 10 million hits a day from an architecture of this sort \u2014 so if you grow further, you\u2019ll need to start adding redundancy.<\/p>\n

This is a good thing, actually. One glance at Figure 20-3 shows you that if even a single one of your three servers fails, you\u2019ll bring down your entire site. So as you add redundant servers, not only do you increase capacity, but you also increase reliability.<\/p>\n

It\u2019s relatively easy to get multiple copies of a Django site running on different hardware \u2013 just copy all the code onto multiple machines, and start Apache on all of them. However, you\u2019ll need another piece of software to distribute traffic over your multiple servers: a load balancer.<\/span><\/p>\n

 <\/span>You can buy expensive and proprietary hardware load balancers, but there are a few high-quality open source software load balancers out there. Apache\u2019s mod_proxy is one option, but we\u2019ve found Perlbal to be fantastic. It\u2019s a load balancer and reverse proxy written by the same folks who wrote memcached.<\/span><\/p>\n

With the Web servers now clustered, our evolving architecture starts to look more complex, as shown in Figure 20-4.<\/p>\n

\"image035\"<\/p>\n

Figure 20-4: A load-balanced, redundant server setup.<\/p>\n

 <\/p>\n

Notice that in the diagram the Web servers are referred to as a \u201ccluster\u201d to indicate that the number of servers is basically variable. Once you have a load balancer out front, you can easily add and remove back-end Web servers without a second of downtime.<\/p>\n

Going Big<\/strong><\/p>\n

At this point, the next few steps are pretty much derivatives of the last one:<\/p>\n