Adds a jobs queue to Payload.
- [x] Docs, w/ examples for Vercel Cron, additional services
- [x] Type the `job` using GeneratedTypes in `JobRunnerArgs`
(@AlessioGr)
- [x] Write the `runJobs` function
- [x] Allow for some type of `payload.runTask`
- [x] Open up a new bin script for running jobs
- [x] Determine strategy for runner endpoint to either await jobs
successfully or return early and stay open until job work completes
(serverless ramifications here)
- [x] Allow for job runner to accept how many jobs to run in one
invocation
- [x] Make a Payload local API method for creating a new job easily
(payload.createJob) or similar which is strongly typed (@AlessioGr)
- [x] Make `payload.runJobs` or similar (@AlessioGr)
- [x] Write tests for retrying up to max retries for a given step
- [x] Write tests for dynamic import of a runner
The shape of the config should permit the definition of steps separate
from the job workflows themselves.
```js
const config = {
// Not sure if we need this property anymore
queues: {
},
// A job is an instance of a workflow, stored in DB
// and triggered by something at some point
jobs: {
// Be able to override the jobs collection
collectionOverrides: () => {},
// Workflows are groups of tasks that handle
// the flow from task to task.
// When defined on the config, they are considered as predefined workflows
// BUT - in the future, we'll allow for UI-based workflow definition as well.
workflows: [
{
slug: 'job-name',
// Temporary name for this
// should be able to pass function
// or path to it for Node to dynamically import
controlFlowInJS: '/my-runner.js',
// Temporary name as well
// should be able to eventually define workflows
// in UI (meaning they need to be serialized in JSON)
// Should not be able to define both control flows
controlFlowInJSON: [
{
task: 'myTask',
next: {
// etc
}
}
],
// Workflows take input
// which are a group of fields
input: [
{
name: 'post',
type: 'relationship',
relationTo: 'posts',
maxDepth: 0,
required: true,
},
{
name: 'message',
type: 'text',
required: true,
},
],
},
],
// Tasks are defined separately as isolated functions
// that can be retried on fail
tasks: [
{
slug: 'myTask',
retries: 2,
// Each task takes input
// Used to auto-type the task func args
input: [
{
name: 'post',
type: 'relationship',
relationTo: 'posts',
maxDepth: 0,
required: true,
},
{
name: 'message',
type: 'text',
required: true,
},
],
// Each task takes output
// Used to auto-type the function signature
output: [
{
name: 'success',
type: 'checkbox',
}
],
onSuccess: () => {},
onFail: () => {},
run: myRunner,
},
]
}
}
```
### `payload.createJob`
This function should allow for the creation of jobs based on either a
workflow (group of tasks) or an individual task.
To create a job using a workflow:
```js
const job = await payload.createJob({
// Accept the `name` of a workflow so we can match to either a
// code-based workflow OR a workflow defined in the DB
// Should auto-type the input
workflowName: 'myWorkflow',
input: {
// typed to the args of the workflow by name
}
})
```
To create a job using a task:
```js
const job = await payload.createJob({
// Accept the `name` of a task
task: 'myTask',
input: {
// typed to the args of the task by name
}
})
```
---------
Co-authored-by: Alessio Gravili <alessio@gravili.de>
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
446 lines
9.5 KiB
TypeScript
446 lines
9.5 KiB
TypeScript
/* tslint:disable */
|
|
/* eslint-disable */
|
|
/**
|
|
* This file was automatically generated by Payload.
|
|
* DO NOT MODIFY IT BY HAND. Instead, modify your source Payload config,
|
|
* and re-run `payload generate:types` to regenerate this file.
|
|
*/
|
|
|
|
export interface Config {
|
|
auth: {
|
|
users: UserAuthOperations;
|
|
};
|
|
collections: {
|
|
posts: Post;
|
|
simple: Simple;
|
|
users: User;
|
|
'payload-jobs': PayloadJob;
|
|
'payload-locked-documents': PayloadLockedDocument;
|
|
'payload-preferences': PayloadPreference;
|
|
'payload-migrations': PayloadMigration;
|
|
};
|
|
db: {
|
|
defaultIDType: string;
|
|
};
|
|
globals: {};
|
|
locale: null;
|
|
user: User & {
|
|
collection: 'users';
|
|
};
|
|
jobs?: {
|
|
tasks: {
|
|
UpdatePost: MyUpdatePostType;
|
|
UpdatePostStep2: TaskUpdatePostStep2;
|
|
CreateSimple: TaskCreateSimple;
|
|
CreateSimpleWithDuplicateMessage: TaskCreateSimpleWithDuplicateMessage;
|
|
ExternalTask: TaskExternalTask;
|
|
inline?: {
|
|
input: unknown;
|
|
output: unknown;
|
|
};
|
|
};
|
|
workflows?: {
|
|
updatePost?: MyUpdatePostWorkflowType;
|
|
updatePostJSONWorkflow?: WorkflowUpdatePostJSONWorkflow;
|
|
retriesTest?: WorkflowRetriesTest;
|
|
retriesRollbackTest?: WorkflowRetriesRollbackTest;
|
|
retriesWorkflowLevelTest?: WorkflowRetriesWorkflowLevelTest;
|
|
inlineTaskTest?: WorkflowInlineTaskTest;
|
|
externalWorkflow?: WorkflowExternalWorkflow;
|
|
retriesBackoffTest?: WorkflowRetriesBackoffTest;
|
|
};
|
|
};
|
|
}
|
|
export interface UserAuthOperations {
|
|
forgotPassword: {
|
|
email: string;
|
|
password: string;
|
|
};
|
|
login: {
|
|
email: string;
|
|
password: string;
|
|
};
|
|
registerFirstUser: {
|
|
email: string;
|
|
password: string;
|
|
};
|
|
unlock: {
|
|
email: string;
|
|
password: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "posts".
|
|
*/
|
|
export interface Post {
|
|
id: string;
|
|
title: string;
|
|
content?: {
|
|
root: {
|
|
type: string;
|
|
children: {
|
|
type: string;
|
|
version: number;
|
|
[k: string]: unknown;
|
|
}[];
|
|
direction: ('ltr' | 'rtl') | null;
|
|
format: 'left' | 'start' | 'center' | 'right' | 'end' | 'justify' | '';
|
|
indent: number;
|
|
version: number;
|
|
};
|
|
[k: string]: unknown;
|
|
} | null;
|
|
jobStep1Ran?: string | null;
|
|
jobStep2Ran?: string | null;
|
|
updatedAt: string;
|
|
createdAt: string;
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "simple".
|
|
*/
|
|
export interface Simple {
|
|
id: string;
|
|
title: string;
|
|
updatedAt: string;
|
|
createdAt: string;
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "users".
|
|
*/
|
|
export interface User {
|
|
id: string;
|
|
updatedAt: string;
|
|
createdAt: string;
|
|
email: string;
|
|
resetPasswordToken?: string | null;
|
|
resetPasswordExpiration?: string | null;
|
|
salt?: string | null;
|
|
hash?: string | null;
|
|
loginAttempts?: number | null;
|
|
lockUntil?: string | null;
|
|
password?: string | null;
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "payload-jobs".
|
|
*/
|
|
export interface PayloadJob {
|
|
id: string;
|
|
input?:
|
|
| {
|
|
[k: string]: unknown;
|
|
}
|
|
| unknown[]
|
|
| string
|
|
| number
|
|
| boolean
|
|
| null;
|
|
taskStatus?:
|
|
| {
|
|
[k: string]: unknown;
|
|
}
|
|
| unknown[]
|
|
| string
|
|
| number
|
|
| boolean
|
|
| null;
|
|
completedAt?: string | null;
|
|
totalTried?: number | null;
|
|
hasError?: boolean | null;
|
|
error?:
|
|
| {
|
|
[k: string]: unknown;
|
|
}
|
|
| unknown[]
|
|
| string
|
|
| number
|
|
| boolean
|
|
| null;
|
|
log?:
|
|
| {
|
|
executedAt: string;
|
|
completedAt: string;
|
|
taskSlug:
|
|
| 'inline'
|
|
| 'UpdatePost'
|
|
| 'UpdatePostStep2'
|
|
| 'CreateSimple'
|
|
| 'CreateSimpleWithDuplicateMessage'
|
|
| 'ExternalTask';
|
|
taskID: string;
|
|
input?:
|
|
| {
|
|
[k: string]: unknown;
|
|
}
|
|
| unknown[]
|
|
| string
|
|
| number
|
|
| boolean
|
|
| null;
|
|
output?:
|
|
| {
|
|
[k: string]: unknown;
|
|
}
|
|
| unknown[]
|
|
| string
|
|
| number
|
|
| boolean
|
|
| null;
|
|
state: 'failed' | 'succeeded';
|
|
error?:
|
|
| {
|
|
[k: string]: unknown;
|
|
}
|
|
| unknown[]
|
|
| string
|
|
| number
|
|
| boolean
|
|
| null;
|
|
id?: string | null;
|
|
}[]
|
|
| null;
|
|
workflowSlug?:
|
|
| (
|
|
| 'updatePost'
|
|
| 'updatePostJSONWorkflow'
|
|
| 'retriesTest'
|
|
| 'retriesRollbackTest'
|
|
| 'retriesWorkflowLevelTest'
|
|
| 'inlineTaskTest'
|
|
| 'externalWorkflow'
|
|
| 'retriesBackoffTest'
|
|
)
|
|
| null;
|
|
taskSlug?:
|
|
| (
|
|
| 'inline'
|
|
| 'UpdatePost'
|
|
| 'UpdatePostStep2'
|
|
| 'CreateSimple'
|
|
| 'CreateSimpleWithDuplicateMessage'
|
|
| 'ExternalTask'
|
|
)
|
|
| null;
|
|
queue?: 'default' | null;
|
|
waitUntil?: string | null;
|
|
processing?: boolean | null;
|
|
updatedAt: string;
|
|
createdAt: string;
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "payload-locked-documents".
|
|
*/
|
|
export interface PayloadLockedDocument {
|
|
id: string;
|
|
document?:
|
|
| ({
|
|
relationTo: 'posts';
|
|
value: string | Post;
|
|
} | null)
|
|
| ({
|
|
relationTo: 'simple';
|
|
value: string | Simple;
|
|
} | null)
|
|
| ({
|
|
relationTo: 'users';
|
|
value: string | User;
|
|
} | null)
|
|
| ({
|
|
relationTo: 'payload-jobs';
|
|
value: string | PayloadJob;
|
|
} | null);
|
|
globalSlug?: string | null;
|
|
user: {
|
|
relationTo: 'users';
|
|
value: string | User;
|
|
};
|
|
updatedAt: string;
|
|
createdAt: string;
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "payload-preferences".
|
|
*/
|
|
export interface PayloadPreference {
|
|
id: string;
|
|
user: {
|
|
relationTo: 'users';
|
|
value: string | User;
|
|
};
|
|
key?: string | null;
|
|
value?:
|
|
| {
|
|
[k: string]: unknown;
|
|
}
|
|
| unknown[]
|
|
| string
|
|
| number
|
|
| boolean
|
|
| null;
|
|
updatedAt: string;
|
|
createdAt: string;
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "payload-migrations".
|
|
*/
|
|
export interface PayloadMigration {
|
|
id: string;
|
|
name?: string | null;
|
|
batch?: number | null;
|
|
updatedAt: string;
|
|
createdAt: string;
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "MyUpdatePostType".
|
|
*/
|
|
export interface MyUpdatePostType {
|
|
input: {
|
|
post: string | Post;
|
|
message: string;
|
|
};
|
|
output: {
|
|
messageTwice: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "TaskUpdatePostStep2".
|
|
*/
|
|
export interface TaskUpdatePostStep2 {
|
|
input: {
|
|
post: string | Post;
|
|
messageTwice: string;
|
|
};
|
|
output?: unknown;
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "TaskCreateSimple".
|
|
*/
|
|
export interface TaskCreateSimple {
|
|
input: {
|
|
message: string;
|
|
shouldFail?: boolean | null;
|
|
};
|
|
output: {
|
|
simpleID: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "TaskCreateSimpleWithDuplicateMessage".
|
|
*/
|
|
export interface TaskCreateSimpleWithDuplicateMessage {
|
|
input: {
|
|
message: string;
|
|
shouldFail?: boolean | null;
|
|
};
|
|
output: {
|
|
simpleID: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "TaskExternalTask".
|
|
*/
|
|
export interface TaskExternalTask {
|
|
input: {
|
|
message: string;
|
|
};
|
|
output: {
|
|
simpleID: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "MyUpdatePostWorkflowType".
|
|
*/
|
|
export interface MyUpdatePostWorkflowType {
|
|
input: {
|
|
post: string | Post;
|
|
message: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "WorkflowUpdatePostJSONWorkflow".
|
|
*/
|
|
export interface WorkflowUpdatePostJSONWorkflow {
|
|
input: {
|
|
post: string | Post;
|
|
message: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "WorkflowRetriesTest".
|
|
*/
|
|
export interface WorkflowRetriesTest {
|
|
input: {
|
|
message: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "WorkflowRetriesRollbackTest".
|
|
*/
|
|
export interface WorkflowRetriesRollbackTest {
|
|
input: {
|
|
message: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "WorkflowRetriesWorkflowLevelTest".
|
|
*/
|
|
export interface WorkflowRetriesWorkflowLevelTest {
|
|
input: {
|
|
message: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "WorkflowInlineTaskTest".
|
|
*/
|
|
export interface WorkflowInlineTaskTest {
|
|
input: {
|
|
message: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "WorkflowExternalWorkflow".
|
|
*/
|
|
export interface WorkflowExternalWorkflow {
|
|
input: {
|
|
message: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "WorkflowRetriesBackoffTest".
|
|
*/
|
|
export interface WorkflowRetriesBackoffTest {
|
|
input: {
|
|
message: string;
|
|
};
|
|
}
|
|
/**
|
|
* This interface was referenced by `Config`'s JSON-Schema
|
|
* via the `definition` "auth".
|
|
*/
|
|
export interface Auth {
|
|
[k: string]: unknown;
|
|
}
|
|
|
|
|
|
declare module 'payload' {
|
|
// @ts-ignore
|
|
export interface GeneratedTypes extends Config {}
|
|
} |