task-queues.html

<!DOCTYPE html>
<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta name="description" content="Full Stack Python shows how an entire Python web application is built and deployed. Each section of the guide explains a different key concept, from the server through the Python WSGI web framework to the front end JavaScript.">
    <meta name="author" content="Matt Makai">
    <link rel="shortcut icon" href="theme/img/full-stack-python-logo-bw.png">
    <title>Full Stack Python</title>
    <!-- Bootstrap core CSS -->
    <link href="theme/css/fsp.css" rel="stylesheet">
    <!--[if lt IE 9]>
      <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
      <script src="https://oss.maxcdn.com/libs/respond.js/1.3.0/respond.min.js"></script>
    <![endif]-->
    <style>
    html,
    body {
      font-size: 18px;
      color: #222;
      background: #fefefe;
    }
    body {
      padding-top: 30px;
    }
    .footer {
      padding: 20px 0 30px 0;
    }
    a, a:hover {border-bottom: 1px dotted; color: #444;}
    a:hover {text-decoration: none; color: #000;}
   .logo-title {font-size: 56px; color: #403072; padding-top: 80px;
                font-family: "News Cycle", "Arial Narrow Bold", sans-serif;
                font-weight: bold; line-height: 30px; margin-left: 5px;}
    .logo-title a, .logo-title a:hover {color: #000; text-decoration: none;
                                        border-bottom: none;}
    .logo-title a:hover {color: gray;}
    .logo-image {vertical-align: top; border: none;}
    a.list-group-item.active {background: #444; border: 1px solid #222;}
    a.list-group-item.active:hover {background: #444; border: 1px solid #222;}
    #sidebar {margin-top: 30px;}

    @media (max-width: 600px) {
        .logo-header-section {
          margin: 20px 32px 0 0;
        }
    }
    </style>
        
    <script type="text/javascript">
      var _gaq = _gaq || [];
      _gaq.push(['_setAccount', 'UA-19910497-7']);
  _gaq.push(['_trackPageview']);

  (function() {
    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
  })();
    </script>
  </head>

<body>
    <a href="https://github.com/makaimc/fullstackpython.github.com" class="github">
        <img style="position: absolute; top: 0; right: 0; border: 0;" src="http://s3.amazonaws.com/github/ribbons/forkme_right_darkblue_121621.png" alt="Fork me on GitHub" />
    </a>
    <div class="container">
        <div class="row">
    <div class="col-md-12">
        <div class="logo-header-section">
            <a href="/" style="text-decoration: none; border: none;"><img src="theme/img/full-stack-python-logo-bw.png" height="42" width="42" class="logo-image" /></a>
            <span class="logo-title"><a href="/">Full Stack Python</a></span>
        </div>
    </div>
</div>
        <div class="row">
    <div class="col-md-8">
      <h1>Task Queues</h1>
      <p>Task queues handle background work that need to be processed outside the
usual HTTP request-response cycle. These tasks are handled asynchronously
because HTTP requests must be responded back to by the server as fast as
possible otherwise the user experience in the web browser will suffer. The
most common types of jobs for task queues include</p>
<ul class="simple">
<li>calculating computationally expensive data analytics</li>
<li>scheduling periodic jobs such as batch processes</li>
<li>spreading out large numbers of independent database inserts over time
instead of all at once</li>
<li>aggregating collected data values on a fixed interval, such as every
15 minutes</li>
</ul>
<div class="section" id="task-queue-resources">
<h2>Task Queue resources</h2>
<p><a class="reference external" href="http://queues.io/">Queues.io</a> is a collection of task queue systems with
short summaries for each one. The task queues are not all compatible with
Python but ones that work with it are tagged with the &quot;Python&quot; keyword.</p>
<p>The <a class="reference external" href="http://www.celeryproject.org/">Celery</a> distributed task queue is the
most commonly used Python library for handling asynchronous tasks and
scheduling.</p>
<p>The <a class="reference external" href="http://python-rq.org/">RQ (Redis Queue)</a> is a simple Python
library for queueing jobs and processing them in the background with workers.
It is backed by Redis and it is designed to have a low barrier to entry.
It should be integrated in your web stack easily.</p>
</div>

                                                                                                                                                                                                                                                    <br/>
            Next read the 
            <a href="/static-content.html">static content</a> section.
                
                                                                                                                                                                                                                                                                                                  </div>
    <div class="col-md-offset-1 col-md-3" id="sidebar">
  <div class="list-group">
          <a href="/introduction.html" class="list-group-item ">Introduction</a>
          <a href="/servers.html" class="list-group-item ">Servers</a>
          <a href="/operating-systems.html" class="list-group-item ">Operating Systems</a>
          <a href="/web-servers.html" class="list-group-item ">Web Servers</a>
          <a href="/platform-as-a-service.html" class="list-group-item ">Platform-as-a-service</a>
          <a href="/databases.html" class="list-group-item ">Databases</a>
          <a href="/wsgi-servers.html" class="list-group-item ">WSGI Servers</a>
          <a href="/web-frameworks.html" class="list-group-item ">Web Frameworks</a>
          <a href="/task-queues.html" class="list-group-item active">Task Queues</a>
          <a href="/static-content.html" class="list-group-item ">Static Content</a>
          <a href="/caching.html" class="list-group-item ">Caching</a>
          <a href="/web-browsers.html" class="list-group-item ">Web Browsers</a>
          <a href="/web-application-security.html" class="list-group-item ">Web Application Security</a>
          <a href="/monitoring.html" class="list-group-item ">Monitoring</a>
          <a href="/web-analytics.html" class="list-group-item ">Web Analytics</a>
          <a href="/api-integration.html" class="list-group-item ">API Integration</a>
          <a href="/source-control.html" class="list-group-item ">Source Control</a>
          <a href="/configuration-management.html" class="list-group-item ">Configuration Management</a>
          <a href="/dependency-management.html" class="list-group-item ">Application Dependencies</a>
          <a href="/no-sql-datastore.html" class="list-group-item ">NoSQL Data Stores</a>
          <a href="/about-author.html" class="list-group-item ">About</a>
          <a href="/change-log.html" class="list-group-item ">Change Log</a>
      </div>
</div></div>
        <hr/>
        <div class="footer pull-right">
            <a href="http://www.mattmakai.com/" class="underline">Matt Makai</a> 2014
        </div>
    </div>

    <script src="http://code.jquery.com/jquery-2.1.0.min.js"></script>
    <script src="theme/js/bootstrap.min.js"></script>
    </body>
</html>