Previously, a single run of the simplest job queue workflow (1 single
task, no db calls by user code in the task - we're just testing db
system overhead) would result in **22 db roundtrips** on drizzle. This
PR reduces it to **17 db roundtrips** by doing the following:
- Modifies db.updateJobs to use the new optimized upsertRow function if
the update is simple
- Do not unnecessarily pass the job log to the final job update when the
workflow completes => allows using the optimized upsertRow function, as
only the main table is involved
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1210888186878606
Previously, `db.deleteMany` on postgres resulted in 2 roundtrips to the
database (find + delete with ids). This PR passes the where query
directly to the `deleteWhere` function, resulting in only one roundtrip
to the database (delete with where).
If the where query queries other tables (=> joins required), this falls
back to find + delete with ids. However, this is also more optimized
than before, as we now pass `select: { id: true }` to the findMany
query.
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1210871676349299
### What?
This PR introduces complete trash (soft-delete) support. When a
collection is configured with `trash: true`, documents can now be
soft-deleted and restored via both the API and the admin panel.
```
import type { CollectionConfig } from 'payload'
const Posts: CollectionConfig = {
slug: 'posts',
trash: true, // <-- New collection config prop @default false
fields: [
{
name: 'title',
type: 'text',
},
// other fields...
],
}
```
### Why
Soft deletes allow developers and admins to safely remove documents
without losing data immediately. This enables workflows like reversible
deletions, trash views, and auditing—while preserving compatibility with
drafts, autosave, and version history.
### How?
#### Backend
- Adds new `trash: true` config option to collections.
- When enabled:
- A `deletedAt` timestamp is conditionally injected into the schema.
- Soft deletion is performed by setting `deletedAt` instead of removing
the document from the database.
- Extends all relevant API operations (`find`, `findByID`, `update`,
`delete`, `versions`, etc.) to support a new `trash` param:
- `trash: false` → excludes trashed documents (default)
- `trash: true` → includes both trashed and non-trashed documents
- To query **only trashed** documents: use `trash: true` with a `where`
clause like `{ deletedAt: { exists: true } }`
- Enforces delete access control before allowing a soft delete via
update or updateByID.
- Disables version restoring on trashed documents (must be restored
first).
#### Admin Panel
- Adds a dedicated **Trash view**: `/collections/:collectionSlug/trash`
- Default delete action now soft-deletes documents when `trash: true` is
set.
- **Delete confirmation modal** includes a checkbox to permanently
delete instead.
- Trashed documents:
- Displays UI banner for better clarity of trashed document edit view vs
non-trashed document edit view
- Render in a read-only edit view
- Still allow access to **Preview**, **API**, and **Versions** tabs
- Updated Status component:
- Displays “Previously published” or “Previously a draft” for trashed
documents.
- Disables status-changing actions when documents are in trash.
- Adds new **Restore** bulk action to clear the `deletedAt` timestamp.
- New `Restore` and `Permanently Delete` buttons for
single-trashed-document restore and permanent deletion.
- **Restore confirmation modal** includes a checkbox to restore as
`published`, defaults to `draft`.
- Adds **Empty Trash** and **Delete permanently** bulk actions.
#### Notes
- This feature is completely opt-in. Collections without trash: true
behave exactly as before.
https://github.com/user-attachments/assets/00b83f8a-0442-441e-a89e-d5dc1f49dd37
~~Sometimes, drizzle is adding the same join to the joins array twice
(`addJoinTable`), despite the table being the same. This is due to a bug
in `getNameFromDrizzleTable` where it would sometimes return a UUID
instead of the table name.~~
~~This PR changes it to read from the drizzle:BaseName symbol instead,
which is correctly returning the table name in my testing. It falls back
to `getTableName`, which uses drizzle:Name.~~
This for some reason fails the tests. Instead, this PR just uses the
getTableName utility now instead of searching for the symbol manually.
Currently, an optimized DB update (simple data => no
delete-and-create-row) does the following:
1. sql UPDATE
2. sql SELECT
This PR reduces this further to one single DB call for simple
collections:
1. sql UPDATE with RETURNING()
This only works for simple collections that do not have any fields that
need to be fetched from other tables. If a collection has fields like
relationship or blocks, we'll need that separate SELECT call to join in
the other tables.
In 4.0, we can remove all "complex" fields from the jobs collection and
replace them with a JSON field to make use of this optimization
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1210803039809814
Adds a new operation findDistinct that can give you distinct values of a
field for a given collection
Example:
Assume you have a collection posts with multiple documents, and some of
them share the same title:
```js
// Example dataset (some titles appear multiple times)
[
{ title: 'title-1' },
{ title: 'title-2' },
{ title: 'title-1' },
{ title: 'title-3' },
{ title: 'title-2' },
{ title: 'title-4' },
{ title: 'title-5' },
{ title: 'title-6' },
{ title: 'title-7' },
{ title: 'title-8' },
{ title: 'title-9' },
]
```
You can now retrieve all unique title values using findDistinct:
```js
const result = await payload.findDistinct({
collection: 'posts',
field: 'title',
})
console.log(result.values)
// Output:
// [
// 'title-1',
// 'title-2',
// 'title-3',
// 'title-4',
// 'title-5',
// 'title-6',
// 'title-7',
// 'title-8',
// 'title-9'
// ]
```
You can also limit the number of distinct results:
```js
const limitedResult = await payload.findDistinct({
collection: 'posts',
field: 'title',
sortOrder: 'desc',
limit: 3,
})
console.log(limitedResult.values)
// Output:
// [
// 'title-1',
// 'title-2',
// 'title-3'
// ]
```
You can also pass a `where` query to filter the documents.
This is a follow-up to https://github.com/payloadcms/payload/pull/13060.
There are a bunch of other db adapter methods that use `upsertRow` for
updates: `updateGlobal`, `updateGlobalVersion`, `updateJobs`,
`updateMany`, `updateVersion`.
The previous PR had the logic for using the optimized row updating logic
inside the `updateOne` adapter. This PR moves that logic to the original
`upsertRow` function. Benefits:
- all the other db methods will benefit from this massive optimization
as well. This will be especially relevant for optimizing postgres job
queue initial updates - we should be able to close
https://github.com/payloadcms/payload/pull/11865 after another follow-up
PR
- easier to read db adapter methods due to less code.
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1210803039809810
Based on https://github.com/payloadcms/payload/pull/13060 which should
be merged first
This PR adds ability to update number fields atomically, which could be
important with parallel writes. For now we support this only via
`payload.db.updateOne`.
For example:
```js
// increment by 10
const res = await payload.db.updateOne({
data: {
number: {
$inc: 10,
},
},
collection: 'posts',
where: { id: { equals: post.id } },
})
// decrement by 3
const res2 = await payload.db.updateOne({
data: {
number: {
$inc: -3,
},
},
collection: 'posts',
where: { id: { equals: post.id } },
})
```
In case, if `payload.db.updateOne` received simple data, meaning no:
* Arrays / Blocks
* Localized Fields
* `hasMany: true` text / select / number / relationship fields
* relationship fields with `relationTo` as an array
This PR simplifies the logic to a single SQL `set` call. No any extra
(useless) steps with rewriting all the arrays / blocks / localized
tables even if there were no any changes to them. However, it's good to
note that `payload.update` (not `payload.db.updateOne`) as for now
passes all the previous data as well, so this change won't have any
effect unless you're using `payload.db.updateOne` directly (or for our
internal logic that uses it), in the future a separate PR with
optimization for `payload.update` as well may be implemented.
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1210710489889576
Previously, `db.updateOne` calls with `where` queries that lead to no
results would create new rows on drizzle. Essentially, `db.updateOne`
behaved like `db.upsertOne` on drizzle
Adds support for `halfvec` and `sparsevec` and `bit` (binary vector)
column types. This is required for supporting indexing of embeddings >
2000 dimensions on postgres using the pg-vector extension.
Continuation of https://github.com/payloadcms/payload/pull/6245.
This PR allows you to pass `blocksAsJSON: true` to SQL adapters and the
adapter instead of aligning with the SQL preferred relation approach for
blocks will just use a simple JSON column, which can improve performance
with a large amount of blocks.
To try these changes you can install `3.43.0-internal.c5bbc84`.
Fixes https://github.com/payloadcms/payload/issues/12628
When using sqlite, the error from the db is a bit different than
Postgres.
This PR allows us to extract the fieldName when using sqlite for the
unique constraint error.
Adds support for read replicas
https://orm.drizzle.team/docs/read-replicas that can be used to offload
read-heavy traffic.
To use (both `db-postgres` and `db-vercel-postgres` are supported):
```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
database: postgresAdapter({
pool: {
connectionString: process.env.POSTGRES_URL,
},
readReplicas: [process.env.POSTGRES_REPLICA_URL],
})
```
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
### What?
This PR fixes an issue while using `text` & `number` fields with
`hasMany: true` where the last entry would be unreachable, and thus
undeletable, because the `transformForWrite` function did not track
these rows for deletion. This causes values that should've been deleted
to remain in the edit view form, as well as the db, after a submission.
This PR also properly threads the placeholder value from
`admin.placeholder` to `text` & `number` `hasMany: true` fields.
### Why?
To remove rows from the db when a submission is made where these fields
are empty arrays, and to properly show an appropriate placeholder when
one is set in config.
### How?
Adjusting `transformForWrite` and the `traverseFields` to keep track of
rows for deletion.
Fixes#11781
Before:
[Editing---Post-dbpg-before--Payload.webm](https://github.com/user-attachments/assets/5ba1708a-2672-4b36-ac68-05212f3aa6cb)
After:
[Editing---Post--dbpg-hasmany-after-Payload.webm](https://github.com/user-attachments/assets/1292e998-83ff-49d0-aa86-6199be319937)
Previously, this was possible in MongoDB but not in Postgres/SQLite
(having `null` in an `in` query)
```
const { docs } = await payload.find({
collection: 'posts',
where: { text: { in: ['text-1', 'text-3', null] } },
})
```
This PR fixes that behavior
The databases do not keep track of document order internally so when
sorting by non-unique fields such as shared `order` number values, the
returned order will be random and not consistent.
While this issue is far more noticeable on mongo it could also occur in
postgres on certain environments.
This combined with pagination can lead to the perception of duplicated
or inconsistent data.
This PR adds a second sort parameter to queries so that we always have a
fallback, `-createdAt` will be used by default or `id` if timestamps are
disabled.
⚠️ `orderable` fields will no longer be `required` and `unique`, so your
database may prompt you to accept an automatic migration if you're using
[this
feature](https://payloadcms.com/docs/configuration/collections#config-options).
Note that the `orderable` feature is still experimental, so it may still
receive breaking changes without a major upgrade or contain bugs. Use it
with caution.
___
The `orderable` fields will not have `required` and `unique` constraints
at the database schema level, in order to automatically migrate
collections that incorporate this property.
Now, when a user adds the `orderable` property to a collection or join
field, existing documents will have the order field set to undefined.
The first time you try to reorder them, the documents will be
automatically assigned an initial order, and you will be prompted to
refresh the page.
We believe this provides a better development experience than having to
manually migrate data with a script.
Additionally, it fixes a bug that occurred when using `orderable` in
conjunction with groups and tabs fields.
Closes:
- #12129
- #12331
- #12212
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
Continuation of https://github.com/payloadcms/payload/pull/12185 and fix
https://github.com/payloadcms/payload/issues/12221
The mentioned PR introduced auto sorting by the point field when a
`near` query is used, but it didn't build actual needed query to order
results by their distance to a _given_ (from the `near` query) point.
Now, we build:
```sql
order by pont_field <-> ST_SetSRID(ST_MakePoint(lng, lat), 4326)
```
Which does what we want
This fixes issues identified in the predefined migration for
postgres v2-v3 including the following:
### relation already exists
Can error with the following:
```ts
{
err: [DatabaseError],
msg: 'Error running migration 20250502_020052_relationships_v2_v3 column "relation_id" of relation "table_name" already exists.'
}
```
This was happening when you run a migration with both a required
relationship or upload field and no schema specified in the db adapter.
When both of these are true the function that replaces `ADD COLUMN` and
`ALTER COLUMN` in order to add `NOT NULL` constraints for requried
fields, wasn't working. This resulted in the `ADD COLUMN` statement from
being being called multiple times instead of altering it after data had
been copied over.
### camelCase column change
Enum columns from using `select` or `radio` have changed from camelCase
to snake case in v3. This change was not accounted for in the
relationship migration and needed to be accounted for.
### DROP CONSTRAINT
It was pointed out by
[here](https://github.com/payloadcms/payload/issues/10162#issuecomment-2610018940)
that the `DROP CONSTRAINT` needs to include `IF EXISTS` so that it can
continue if the contraint was already removed in a previous statement.
fixes https://github.com/payloadcms/payload/issues/10162
### What?
Turborepo fails to compile due to type error in the generated drizzle
schema.
### Why?
TypeScript may not include the module augmentation for
@payloadcms/db-postgres, especially in monorepo or isolated module
builds. This causes type errors during the compilation process of
turborepo project. Adding the type-only import guarantees that
TypeScript loads the relevant type definitions and augmentations,
resolving these errors.
### How?
This PR adds a type-only import statement to ensure TypeScript
recognizes the module augmentation for @payloadcms/db-postgres in the
generated drizzle schema from payload, and there is no runtime effect.
Fixes#12311
-->
Fixes https://github.com/payloadcms/payload/issues/12263
This was caused by passing not needed columns to the `SELECT DISTINCT`
query, which we execute in case if we have a filter / sort by a nested
field / relationship. Since the only columns that we need to pass to the
`SELECT DISTINCT` query are: ID and field(s) specified in `sort`, we now
filter the `selectFields` variable.
### What?
Swaps out `deepAssertEqual` for `dequal` package. Further details and
motivation in [this
discussion](https://github.com/payloadcms/payload/discussions/12192).
### Why?
Dequal is about 100x faster in limited local testing. Dequal package
shows 3-5x speed over `deepAssertEqual` in benchmarks. Memory usage is
within acceptable levels.
### How?
Move the result of dequal to a `const` for readability. Replace the `try
{ ... } catch { ... }` with `if { ... } else { ... }` for minimum impact
and change.
When running the v2-v3 migration you might receive prompts for renaming
columns. Since we start a transaction before, you might end up with a
fail if you don't answer within your transaction session period timeout.
This moves the `getTransaction` call after prompts were answered, since
we don't have a reason to start it earlier.
When `payload migrate` is run and a record with name "dev" is returned
having `batch: -1`, then the `batch` is not incrementing as expected as
it is stuck at 1. This change makes it so the batch is incremented from
the correct latest batch, ignoring the `name: "dev"` migration.
This improves performance when querying data in Postgers / SQLite with
`limit: 0`. Before, unless you additionally passed `pagination: false`
we executed additional count query to calculate the pagination. Now we
skip this as this is unnecessary since we can retrieve the count just
from `rows.length`.
This logic already existed in `db-mongodb` -
1b17df9e0b/packages/db-mongodb/src/find.ts (L114-L124)
Fixes https://github.com/payloadcms/payload/issues/12090, in MongoDB the
documents are sorted by distance automatically whenever the `near`
operation is used, which we have in the docs:
> When querying using the near operator, the returned documents will be
sorted by nearest first.
This fixes this incosistensty between Postgres and MongoDB.
⚠️ This change potentially can cause to produce different results, if
you used the `near` operator without `sort: 'pointFieldName'`.
### What?
Converts numbers passed to a text field to avoid the database/drizzle
from converting it incorrectly.
### Why?
If you have a hook that passes a value to another field you can
experience this problem where drizzle converts a number value for a text
field to a floating point number in sqlite for example.
### How?
Adds logic to `transform/write/traverseFields.ts` to cast text field
values to string.
following changes made by Commit a6f7ef8
> feat(db-*): export types from main export (#11914)
In 3.0, we made the decision to export all types from the main package
export (e.g. `payload/types` => `payload`). This improves type
discoverability by IDEs and simplifies importing types.
> This PR does the same for our db adapters, which still have a separate
`/types` subpath export. While those are kept for
backwards-compatibility, we can remove them in 4.0.
a6f7ef837a
the script responsible for generating file generated-schema.ts was not
updated to reflect this change in export paths
drizzle/src/utilities/createSchemaGenerator.ts
CURRENT
```typescript
const finalDeclaration = `
declare module '${this.packageName}/types' {
export interface GeneratedDatabaseSchema {
schema: DatabaseSchema
}
}
```
AFTER THIS PULL REQUEST
```typescript
const finalDeclaration = `
declare module '${this.packageName}' {
export interface GeneratedDatabaseSchema {
schema: DatabaseSchema
}
}
```
this pull request fixes the generation of generated-schema.ts avoiding
errors while building for production with command
```bash
npm run build
```
