### What?
This PR adds atomic array operations ($append and $remove) for
relationship fields with `hasMany: true` across all database adapters.
These operations allow developers to add or remove specific items from
relationship arrays without replacing the entire array.
New API:
```
// Append relationships (prevents duplicates)
await payload.db.updateOne({
collection: 'posts',
id: 'post123',
data: {
categories: { $append: ['featured', 'trending'] }
}
})
// Remove specific relationships
await payload.db.updateOne({
collection: 'posts',
id: 'post123',
data: {
tags: { $remove: ['draft', 'private'] }
}
})
// Works with polymorphic relationships
await payload.db.updateOne({
collection: 'posts',
id: 'post123',
data: {
relatedItems: {
$append: [
{ relationTo: 'categories', value: 'category-id' },
{ relationTo: 'tags', value: 'tag-id' }
]
}
}
})
```
### Why?
Currently, updating relationship arrays requires replacing the entire
array which requires fetching existing data before updates. Requiring
more implementation effort and potential for errors when using the API,
in particular for bulk updates.
### How?
#### Cross-Adapter Features:
- Polymorphic relationships: Full support for relationTo:
['collection1', 'collection2']
- Localized relationships: Proper locale handling when fields are
localized
- Duplicate prevention: Ensures `$append` doesn't create duplicates
- Order preservation: Appends to end of array maintaining order
- Bulk operations: Works with `updateMany` for bulk updates
#### MongoDB Implementation:
- Converts `$append` to native `$addToSet` (prevents duplicates in
contrast to `$push`)
- Converts `$remove` to native `$pull` (targeted removal)
#### Drizzle Implementation (Postgres/SQLite):
- Uses optimized batch `INSERT` with duplicate checking for `$append`
- Uses targeted `DELETE` queries for `$remove`
- Implements timestamp-based ordering for performance
- Handles locale columns conditionally based on schema
### Limitations
The current implementation is only on database-adapter level and not
(yet) for the local API. Implementation in the localAPI will be done
separately.
This PR adds support for the following configuration:
```ts
const config = {
collections: [
{
slug: 'categories',
fields: [
{
name: 'title',
type: 'text',
},
],
},
{
slug: 'posts',
fields: [
{
name: 'title',
type: 'text',
},
{
name: 'categories',
type: 'relationship',
relationTo: 'categories',
hasMany: true,
},
],
},
{
slug: 'examples',
fields: [
{
name: 'postCategoriesTitles',
type: 'text',
virtual: 'post.categories.title',
// hasMany: true - added automatically during the sanitization
},
{
type: 'relationship',
relationTo: 'posts',
name: 'post',
},
{
name: 'postsTitles',
type: 'text',
virtual: 'posts.title',
// hasMany: true - added automatically during the sanitization
},
{
type: 'relationship',
relationTo: 'posts',
name: 'posts',
hasMany: true,
},
],
},
],
}
```
In the result:
`postsTitles` - will be always populated with an array of posts titles.
`postCategoriesTitles` - will be always populated with an array of the
categories titles that are related to this post
The virtual `text` field is sanitizated to `hasMany: true`
automatically, but you can specify that manually as well.
This PR adds **atomic** `$push` **support for array fields**. It makes
it possible to safely append new items to arrays, which is especially
useful when running tasks in parallel (like job queues) where multiple
processes might update the same record at the same time. By handling
pushes atomically, we avoid race conditions and keep data consistent -
especially on postgres, where the current implementation would nuke the
entire array table before re-inserting every single array item.
The feature works for both localized and unlocalized arrays, and
supports pushing either single or multiple items at once.
This PR is a requirement for reliably running parallel tasks in the job
queue - see https://github.com/payloadcms/payload/pull/13452.
Alongside documenting `$push`, this PR also adds documentation for
`$inc`.
## Changes to updatedAt behavior
https://github.com/payloadcms/payload/pull/13335 allows us to override
the updatedAt property instead of the db always setting it to the
current date.
However, we are not able to skip updating the updatedAt property
completely. This means, usage of $push results in 2 postgres db calls:
1. set updatedAt in main row
2. append array row in arrays table
This PR changes the behavior to only automatically set updatedAt if it's
undefined. If you explicitly set it to `null`, this now allows you to
skip the db adapter automatically setting updatedAt.
=> This allows us to use $push in just one single db call
## Usage Examples
### Pushing a single item to an array
```ts
const post = (await payload.db.updateOne({
data: {
array: {
$push: {
text: 'some text 2',
id: new mongoose.Types.ObjectId().toHexString(),
},
},
},
collection: 'posts',
id: post.id,
}))
```
### Pushing a single item to a localized array
```ts
const post = (await payload.db.updateOne({
data: {
arrayLocalized: {
$push: {
en: {
text: 'some text 2',
id: new mongoose.Types.ObjectId().toHexString(),
},
es: {
text: 'some text 2 es',
id: new mongoose.Types.ObjectId().toHexString(),
},
},
},
},
collection: 'posts',
id: post.id,
}))
```
### Pushing multiple items to an array
```ts
const post = (await payload.db.updateOne({
data: {
array: {
$push: [
{
text: 'some text 2',
id: new mongoose.Types.ObjectId().toHexString(),
},
{
text: 'some text 3',
id: new mongoose.Types.ObjectId().toHexString(),
},
],
},
},
collection: 'posts',
id: post.id,
}))
```
### Pushing multiple items to a localized array
```ts
const post = (await payload.db.updateOne({
data: {
arrayLocalized: {
$push: {
en: {
text: 'some text 2',
id: new mongoose.Types.ObjectId().toHexString(),
},
es: [
{
text: 'some text 2 es',
id: new mongoose.Types.ObjectId().toHexString(),
},
{
text: 'some text 3 es',
id: new mongoose.Types.ObjectId().toHexString(),
},
],
},
},
},
collection: 'posts',
id: post.id,
}))
```
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1211110462564647
Previously, when manually setting `createdAt` or `updatedAt` in a
`payload.db.*` or `payload.*` operation, the value may have been
ignored. In some cases it was _impossible_ to change the `updatedAt`
value, even when using direct db adapter calls. On top of that, this
behavior sometimes differed between db adapters. For example, mongodb
did accept `updatedAt` when calling `payload.db.updateVersion` -
postgres ignored it.
This PR changes this behavior to consistently respect `createdAt` and
`updatedAt` values for `payload.db.*` operations.
For `payload.*` operations, this also works with the following
exception:
- update operations do no respect `updatedAt`, as updates are commonly
performed by spreading the old data, e.g. `payload.update({ data:
{...oldData} })` - in these cases, we usually still want the `updatedAt`
to be updated. If you need to get around this, you can use the
`payload.db.updateOne` operation instead.
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1210919646303994
Currently, an optimized DB update (simple data => no
delete-and-create-row) does the following:
1. sql UPDATE
2. sql SELECT
This PR reduces this further to one single DB call for simple
collections:
1. sql UPDATE with RETURNING()
This only works for simple collections that do not have any fields that
need to be fetched from other tables. If a collection has fields like
relationship or blocks, we'll need that separate SELECT call to join in
the other tables.
In 4.0, we can remove all "complex" fields from the jobs collection and
replace them with a JSON field to make use of this optimization
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1210803039809814
