You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: Readme.md
+55-54Lines changed: 55 additions & 54 deletions
Original file line number
Diff line number
Diff line change
@@ -64,13 +64,13 @@ First create a job `Queue` with `kue.createQueue()`:
64
64
65
65
```js
66
66
var kue =require('kue')
67
-
, jobs=kue.createQueue();
67
+
, queue=kue.createQueue();
68
68
```
69
69
70
-
Calling `jobs.create()` with the type of job ("email"), and arbitrary job data will return a `Job`, which can then be `save()`ed, adding it to redis, with a default priority level of "normal". The `save()` method optionally accepts a callback, responding with an `error` if something goes wrong. The `title` key is special-cased, and will display in the job listings within the UI, making it easier to find a specific job.
70
+
Calling `queue.create()` with the type of job ("email"), and arbitrary job data will return a `Job`, which can then be `save()`ed, adding it to redis, with a default priority level of "normal". The `save()` method optionally accepts a callback, responding with an `error` if something goes wrong. The `title` key is special-cased, and will display in the job listings within the UI, making it easier to find a specific job.
71
71
72
72
```js
73
-
var job =jobs.create('email', {
73
+
var job =queue.create('email', {
74
74
title:'welcome email for tj'
75
75
, to:'tj@learnboost.com'
76
76
, template:'welcome-email'
@@ -84,7 +84,7 @@ var job = jobs.create('email', {
84
84
To specify the priority of a job, simply invoke the `priority()` method with a number, or priority name, which is mapped to a number.
85
85
86
86
```js
87
-
jobs.create('email', {
87
+
queue.create('email', {
88
88
title:'welcome email for tj'
89
89
, to:'tj@learnboost.com'
90
90
, template:'welcome-email'
@@ -108,7 +108,7 @@ The default priority map is as follows:
108
108
By default jobs only have _one_ attempt, that is when they fail, they are marked as a failure, and remain that way until you intervene. However, Kue allows you to specify this, which is important for jobs such as transferring an email, which upon failure, may usually retry without issue. To do this invoke the `.attempts()` method with a number.
109
109
110
110
```js
111
-
jobs.create('email', {
111
+
queue.create('email', {
112
112
title:'welcome email for tj'
113
113
, to:'tj@learnboost.com'
114
114
, template:'welcome-email'
@@ -171,7 +171,7 @@ Job-specific events are fired on the `Job` instances via Redis pubsub. The follo
171
171
For example this may look something like the following:
Queue-level events provide access to the job-level events previously mentioned, however scoped to the `Queue` instance to apply logic at a "global" level. An example of this is removing completed jobs:
200
200
201
201
```js
202
-
jobs.on('job enqueue', function(id, type){
202
+
queue.on('job enqueue', function(id, type){
203
203
console.log( 'Job %s got queued of type %s', id, type );
204
204
205
205
}).on('job complete', function(id, result){
@@ -221,7 +221,7 @@ Delayed jobs may be scheduled to be queued for an arbitrary distance in time by
221
221
This automatically flags the `Job` as "delayed".
222
222
223
223
```js
224
-
var email =jobs.create('email', {
224
+
var email =queue.create('email', {
225
225
title:'Account renewal required'
226
226
, to:'tj@learnboost.com'
227
227
, template:'renewal-email'
@@ -233,21 +233,21 @@ var email = jobs.create('email', {
233
233
When using delayed jobs, we must also check the delayed jobs with a timer, promoting them if the scheduled delay has been exceeded. This `setInterval` is defined within `Queue#promote(ms,limit)`, defaulting to a check of top 200 jobs every 5 seconds. If you have a cluster of kue processes, you must call `.promote` in just one (preferably master) process or promotion race can happen.
234
234
235
235
```js
236
-
jobs.promote();
236
+
queue.promote();
237
237
```
238
238
239
239
## Processing Jobs
240
240
241
-
Processing jobs is simple with Kue. First create a `Queue` instance much like we do for creating jobs, providing us access to redis etc, then invoke `jobs.process()` with the associated type.
241
+
Processing jobs is simple with Kue. First create a `Queue` instance much like we do for creating jobs, providing us access to redis etc, then invoke `queue.process()` with the associated type.
242
242
Note that unlike what the name `createQueue` suggests, it currently returns a singleton `Queue` instance. So you can configure and use only a single `Queue` object within your node.js process.
243
243
244
244
In the following example we pass the callback `done` to `email`, When an error occurs we invoke `done(err)` to tell Kue something happened, otherwise we invoke `done()` only when the job is complete. If this function responds with an error it will be displayed in the UI and the job will be marked as a failure. The error object passed to done, should be of standard type `Error`.
245
245
246
246
```js
247
247
var kue =require('kue')
248
-
, jobs=kue.createQueue();
248
+
, queue=kue.createQueue();
249
249
250
-
jobs.process('email', function(job, done){
250
+
queue.process('email', function(job, done){
251
251
email(job.data.to, done);
252
252
});
253
253
@@ -265,10 +265,10 @@ Workers can also pass job result as the second parameter to done `done(null,resu
265
265
266
266
### Processing Concurrency
267
267
268
-
By default a call to `jobs.process()` will only accept one job at a time for processing. For small tasks like sending emails this is not ideal, so we may specify the maximum active jobs for this type by passing a number:
268
+
By default a call to `queue.process()` will only accept one job at a time for processing. For small tasks like sending emails this is not ideal, so we may specify the maximum active jobs for this type by passing a number:
Workers can temporary pause and resume their activity. It is, after calling `pause` they will receive no jobs in their process callback until `resume` is called. `pause` function gracefully shutdowns this worker, and uses the same internal functionality as `shutdown` method in [Graceful Shutdown](#graceful-shutdown).
For a "real" example, let's say we need to compile a PDF from numerous slides with [node-canvas](http://github.com/learnboost/node-canvas). Our job may consist of the following data, note that in general you should _not_ store large data in the job it-self, it's better to store references like ids, pulling them in while processing.
292
292
293
293
```js
294
-
jobs.create('slideshow pdf', {
294
+
queue.create('slideshow pdf', {
295
295
title:user.name+"'s slideshow"
296
296
, slides: [...] // keys to data stored in redis, mongodb, or some other store
We can access this same arbitrary data within a separate process while processing, via the `job.data` property. In the example we render each slide one-by-one, updating the job's log and process.
Kue marks a job complete/failed when `done` is called by your worker, so you should use proper error handling to prevent uncaught exceptions in your worker's code and node.js process exiting before in handle jobs get done.
352
352
This can be achieved in two ways:
353
353
354
-
* Wrapping your worker's process function in [Domains](https://nodejs.org/api/domain.html)
354
+
1. Wrapping your worker's process function in [Domains](https://nodejs.org/api/domain.html)
This is the softest and best solution, however is not built-in with Kue. Please refer to [this discussion](https://github.com/kriskowal/q/issues/120). You can comment on this feature in the related open Kue [issue](https://github.com/LearnBoost/kue/pull/403).
369
+
This is the softest and best solution, however is not built-in with Kue. Please refer to [this discussion](https://github.com/kriskowal/q/issues/120). You can comment on this feature in the related open Kue [issue](https://github.com/LearnBoost/kue/pull/403).
Promise.method( function(){ // your process function
376
+
thrownewError( 'bad things happen' );
377
+
})().nodeify(done)
378
+
});
379
+
```
380
380
381
-
but this won't catch exceptions in your async call stack.
381
+
but this won't catch exceptions in your async call stack as domains do.
382
382
383
383
384
-
* Binding to `uncaughtException` and gracefully shutting down the Kue.
385
384
386
-
```js
387
-
process.once( 'uncaughtException', function(err){
388
-
queue.shutdown(function(err2){
389
-
process.exit( 0 );
390
-
}, 2000 );
391
-
});
392
-
```
385
+
2. Binding to `uncaughtException` and gracefully shutting down the Kue.
386
+
387
+
```js
388
+
process.once( 'uncaughtException', function(err){
389
+
queue.shutdown(function(err2){
390
+
process.exit( 0 );
391
+
}, 2000 );
392
+
});
393
+
```
393
394
394
395
### Unstable Redis connections
395
396
396
397
Kue currently uses client side job state management and when redis crashes in the middle of that operations, some stuck jobs or index inconsistencies will happen. If you are facing poor redis connections or an unstable redis service you can start Kue's watchdog to fix stuck inactive jobs (if any) by calling:
397
398
398
399
```js
399
-
jobs.watchStuckJobs()
400
+
queue.watchStuckJobs()
400
401
```
401
402
402
403
Kue will be refactored to fully atomic job state management from version 1.0 and this will happen by lua scripts and/or BRPOPLPUSH combination. You can read more [here](https://github.com/LearnBoost/kue/issues/130) and [here](https://github.com/LearnBoost/kue/issues/38).
@@ -408,7 +409,7 @@ Kue will be refactored to fully atomic job state management from version 1.0 and
408
409
If you did none of above or your process lost active jobs in any way, you can recover from them when your process is restarted. A blind logic would be to re-queue all stuck jobs:
Jobs data and search indexes eat up redis memory space, so you will need some job-keeping process in real world deployments. Your first chance is using automatic job removal on completion.
0 commit comments