Files
payload/docs/database/transactions.mdx
James Mikrut 8970c6b3a6 feat: adds jobs queue (#8228)
Adds a jobs queue to Payload.

- [x] Docs, w/ examples for Vercel Cron, additional services
- [x] Type the `job` using GeneratedTypes in `JobRunnerArgs`
(@AlessioGr)
- [x] Write the `runJobs` function 
- [x] Allow for some type of `payload.runTask` 
- [x] Open up a new bin script for running jobs
- [x] Determine strategy for runner endpoint to either await jobs
successfully or return early and stay open until job work completes
(serverless ramifications here)
- [x] Allow for job runner to accept how many jobs to run in one
invocation
- [x] Make a Payload local API method for creating a new job easily
(payload.createJob) or similar which is strongly typed (@AlessioGr)
- [x] Make `payload.runJobs` or similar  (@AlessioGr)
- [x] Write tests for retrying up to max retries for a given step
- [x] Write tests for dynamic import of a runner

The shape of the config should permit the definition of steps separate
from the job workflows themselves.

```js
const config = {
  // Not sure if we need this property anymore
  queues: {
  },
  // A job is an instance of a workflow, stored in DB
  // and triggered by something at some point
  jobs: {
    // Be able to override the jobs collection
    collectionOverrides: () => {},

    // Workflows are groups of tasks that handle
    // the flow from task to task.
    // When defined on the config, they are considered as predefined workflows
    // BUT - in the future, we'll allow for UI-based workflow definition as well.
    workflows: [
      {
        slug: 'job-name',
        // Temporary name for this
        // should be able to pass function 
        // or path to it for Node to dynamically import
        controlFlowInJS: '/my-runner.js',

        // Temporary name as well
        // should be able to eventually define workflows
        // in UI (meaning they need to be serialized in JSON)
        // Should not be able to define both control flows
        controlFlowInJSON: [
          {
            task: 'myTask',
            next: {
              // etc
            }
          }
        ],

        // Workflows take input
        // which are a group of fields
        input: [
          {
            name: 'post',
            type: 'relationship',
            relationTo: 'posts',
            maxDepth: 0,
            required: true,
          },
          {
            name: 'message',
            type: 'text',
            required: true,
          },
        ],
      },
    ],

    // Tasks are defined separately as isolated functions
    // that can be retried on fail
    tasks: [
      {
        slug: 'myTask',
        retries: 2,
        // Each task takes input
        // Used to auto-type the task func args
        input: [
          {
            name: 'post',
            type: 'relationship',
            relationTo: 'posts',
            maxDepth: 0,
            required: true,
          },
          {
            name: 'message',
            type: 'text',
            required: true,
          },
        ],
        // Each task takes output
        // Used to auto-type the function signature
        output: [
          {
            name: 'success',
            type: 'checkbox',
          }
        ],
        onSuccess: () => {},
        onFail: () => {},
        run: myRunner,
      },
    ]
  }
}
```

### `payload.createJob`

This function should allow for the creation of jobs based on either a
workflow (group of tasks) or an individual task.

To create a job using a workflow:

```js
const job = await payload.createJob({
  // Accept the `name` of a workflow so we can match to either a 
  // code-based workflow OR a workflow defined in the DB
  // Should auto-type the input
  workflowName: 'myWorkflow',
  input: {
    // typed to the args of the workflow by name
  }
})
```

To create a job using a task:

```js
const job = await payload.createJob({
  // Accept the `name` of a task
  task: 'myTask',
  input: {
    // typed to the args of the task by name
  }
})
```

---------

Co-authored-by: Alessio Gravili <alessio@gravili.de>
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2024-10-30 17:56:50 +00:00

132 lines
5.1 KiB
Plaintext

---
title: Transactions
label: Transactions
order: 30
keywords: database, transactions, sql, mongodb, postgres, documentation, Content Management System, cms, headless, typescript, node, react, nextjs
desc: Database transactions are fully supported within Payload.
---
Database transactions allow your application to make a series of database changes in an all-or-nothing commit. Consider an HTTP request that creates a new **Order** and has an `afterChange` hook to update the stock count of related **Items**. If an error occurs when updating an **Item** and an HTTP error is returned to the user, you would not want the new **Order** to be persisted or any other items to be changed either. This kind of interaction with the database is handled seamlessly with transactions.
By default, Payload will use transactions for all data changing operations, as long as it is supported by the configured database. Database changes are contained within all Payload operations and any errors thrown will result in all changes being rolled back without being committed. When transactions are not supported by the database, Payload will continue to operate as expected without them.
<Banner type="info">
<strong>Note:</strong>
<br />
MongoDB requires a connection to a replicaset in order to make use of transactions.
</Banner>
The initial request made to Payload will begin a new transaction and attach it to the `req.transactionID`. If you have a `hook` that interacts with the database, you can opt-in to using the same transaction by passing the `req` in the arguments. For example:
```ts
const afterChange: CollectionAfterChangeHook = async ({ req }) => {
// because req.transactionID is assigned from Payload and passed through,
// my-slug will only persist if the entire request is successful
await req.payload.create({
req,
collection: 'my-slug',
data: {
some: 'data',
},
})
}
```
## Async Hooks with Transactions
Since Payload hooks can be async and be written to not await the result, it is possible to have an incorrect success response returned on a request that is rolled back. If you have a hook where you do not `await` the result, then you should **not** pass the `req.transactionID`.
```ts
const afterChange: CollectionAfterChangeHook = async ({ req }) => {
// WARNING: an async call made with the same req, but NOT awaited,
// may fail resulting in an OK response being returned with response data that is not committed
const dangerouslyIgnoreAsync = req.payload.create({
req,
collection: 'my-slug',
data: {
some: 'other data',
},
})
// Should this call fail, it will not rollback other changes
// because the req (and its transactionID) is not passed through
const safelyIgnoredAsync = req.payload.create({
collection: 'my-slug',
data: {
some: 'other data',
},
})
}
```
## Direct Transaction Access
When writing your own scripts or custom endpoints, you may wish to have direct control over transactions. This is useful for interacting with your database outside of Payload's local API.
The following functions can be used for managing transactions:
`payload.db.beginTransaction` - Starts a new session and returns a transaction ID for use in other Payload Local API calls.
`payload.db.commitTransaction` - Takes the identifier for the transaction, finalizes any changes.
`payload.db.rollbackTransaction` - Takes the identifier for the transaction, discards any changes.
Payload uses the `req` object to pass the transaction ID through to the database adapter. If you are not using the `req` object, you can make a new object to pass the transaction ID directly to database adapter methods and local API calls.
Example:
```ts
import payload from 'payload'
import config from './payload.config'
const standalonePayloadScript = async () => {
// initialize Payload
await payload.init({ config })
const transactionID = await payload.db.beginTransaction()
try {
// Make an update using the local API
await payload.update({
collection: 'posts',
data: {
some: 'data',
},
where: {
slug: { equals: 'my-slug' }
},
req: { transactionID },
})
/*
You can make additional db changes or run other functions
that need to be committed on an all or nothing basis
*/
// Commit the transaction
await payload.db.commitTransaction(transactionID)
} catch (error) {
// Rollback the transaction
await payload.db.rollbackTransaction(transactionID)
}
}
standalonePayloadScript()
```
## Disabling Transactions
If you wish to disable transactions entirely, you can do so by passing `false` as the `transactionOptions` in your database adapter configuration. All the official Payload database adapters support this option.
In addition to allowing database transactions to be disabled at the adapter level. You can prevent Payload from using a transaction in direct calls to the local API by adding `disableTransaction: true` to the args. For example:
```ts
await payload.update({
collection: 'posts',
data: {
some: 'data',
},
where: {
slug: { equals: 'my-slug' }
},
disableTransaction: true,
})
```