### What?
If you had multiple operator constraints on a single field, the last one
defined would be the only one used.
Example:
```ts
where: {
id: {
in: [doc2.id],
not_in: [], // <-- only respected this operator constraint
},
}
```
and
```ts
where: {
id: {
not_in: [],
in: [doc2.id], // <-- only respected this operator constraint
},
}
```
They would yield different results.
### Why?
The results were not merged into an `$and` query inside parseParams.
### How?
Merges the results within an `$and` constraint.
Fixes https://github.com/payloadcms/payload/issues/10944
Supersedes https://github.com/payloadcms/payload/pull/11011
- reduces unnecessary shallow copying within operations by removing
unnecessary spreads or .map()'s
- removes unnecessary `deleteMany` call in `deleteUserPreferences` for
auth-enabled collections
- replaces all instances of `validOperators.includes` with
`validOperatorMap[]`. O(n) => O(1)
- optimizes the `sanitizeInternalFields` function. Previously, it was
doing a **lot** of shallow copying
Previously, every error from MongoDB was logged as "Value must be
unique", as well the response code should not be `BAD_REQUEST` but
`INTERNAL_SERVER_ERROR`. `throw error` preserves the original error so
it can be traced.
### What?
The join field had a limitation imposed that prevents it from targeting
polymorphic relationship fields. With this change we can support any
relationship fields.
### Why?
Improves the functionality of join field.
### How?
Extended the database adapters and removed the config sanitization that
would throw an error when polymorphic relationships were used.
Fixes #
Previously, queries like this didn't work:
```ts
const res = await payload.find({
collection: 'polymorphic-relationships',
where: {
polymorphicLocalized: {
equals: {
relationTo: 'movies',
value: movie.id,
},
},
},
})
```
This was due to the incorrectly passed path to MongoDB without
`.{locale}` suffix.
Additionally, to MongoDB now we send:
```
{
$or: [
{
polymorphic: {
$eq: {
relationTo: formattedValue.relationTo,
value: formattedValue.value,
},
},
},
{
polymorphic: {
$eq: {
relationTo: 'movies',
value: 'some-id',
},
},
},
],
},
```
Instead of:
```
{
$and: [
{
'polymorphic.relationTo': {
$eq: 'movies ',
},
},
{
'polymorphic.value': {
$eq: 'some-id ',
},
},
],
}
```
To match the _exact_ value. This is essential when we do querying by
relationships with `hasMany: true` and custom IDs that can be repeated.
`$or` is needed if for some reason keys are stored in the DB in a
different order
### What?
Previously, the `req` argument:
In database operations (e.g `payload.db`) was required and you needed to
pass the whole `req` with all the properties. This is confusing because
in database operations we never use its properties outside of
`req.transactionID` and `req.t`, both of which should be optional as
well.
Now, you don't have to do that cast:
```ts
payload.db.findOne({
collection: 'posts',
req: {} as PayloadRequest,
where: {
id: {
equals: 1,
},
},
})
```
Becomes:
```ts
payload.db.findOne({
collection: 'posts',
where: {
id: {
equals: 1,
},
},
})
```
If you need to use transactions, you're not required to do the `as` cast
as well now, as the `req` not only optional but also partial -
`Partial<PayloadRequest>`.
`initTransaction`, `commitTransaction`, `killTransaction` utilities are
typed better now as well. They do not require to you pass all the
properties of `req`, but only `payload` -
`MarkRequired<Partial<PayloadRequest>, 'payload'>`
```ts
const req = { payload }
await initTransaction(req)
await payload.db.create({
collection: "posts",
data: {},
req
})
await commitTransaction(req)
```
The same for the Local API. Internal operations (for example
`packages/payload/src/collections/operations/find.ts`) still accept the
whole `req`, but local ones
(`packages/payload/src/collections/operations/local/find.ts`) which are
used through `payload.` now accept `Partial<PayloadRequest>`, as they
pass it through to internal operations with `createLocalReq`.
So now, this is also valid, while previously you had to do `as` cast for
`req`.
```ts
const req = { payload }
await initTransaction(req)
await payload.create({
collection: "posts",
data: {},
req
})
await commitTransaction(req)
```
Marked as deprecated `PayloadRequest['transactionIDPromise']` to remove
in the next major version. It was never used anywhere.
Refactored `withSession` that returns an object to `getSession` that
returns just `ClientSession`. Better type safety for arguments
Deduplicated in all drizzle operations to `getTransaction(this, req)`
utility:
```ts
const db = this.sessions[await req?.transactionID]?.db || this.drizzle
```
Added fallback for throwing unique validation errors in database
operations when `req.t` is not available.
In migration `up` and `down` functions our `req` is not partial, while
we used to passed `req` with only 2 properties - `payload` and
`transactionID`. This is misleading and you can't access for example
`req.t`.
Now, to achieve "real" full `req` - we generate it with `createLocalReq`
in all migration functions.
This all is backwards compatible. In all public API places where you
expect the full `req` (like hooks) you still have it.
### Why?
Better DX, more expected types, less errors because of types casting.
Adds the ability to pass additional schema options for collections with:
```ts
mongooseAdapter({
collectionsSchemaOptions: {
posts: {
strict: false,
},
},
})
```
This changes relates to these:
- https://github.com/payloadcms/payload/issues/4533
- https://github.com/payloadcms/payload/discussions/4534
It is a proposal to set custom schema options for mongoose driver.
I understand this got introduced into `main` v2 after `beta` branch was
created so this feature got lost.
- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
<!--
Thank you for the PR! Please go through the checklist below and make
sure you've completed all the steps.
Please review the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository if you haven't already.
The following items will ensure that your PR is handled as smoothly as
possible:
- PR Title must follow conventional commits format. For example, `feat:
my new feature`, `fix(plugin-seo): my fix`.
- Minimal description explained as if explained to someone not
immediately familiar with the code.
- Provide before/after screenshots or code diffs if applicable.
- Link any related issues/discussions from GitHub or Discord.
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Fixes #
-->
---------
Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
This PR adds a feature which fixes another issue with migrations in
Postgres and does few refactors that significantly reduce code
duplication.
Previously, if you needed to use the underlying database directly in
migrations with the active transaction (for example to execute raw SQL),
created from `payload create:migration`, as `req` doesn't work there you
had to do something like this:
```ts
// Postgres
export async function up({ payload, req }: MigrateUpArgs): Promise<void> {
const db = payload.db.sessions?.[await req.transactionID!].db ?? payload.db
const { rows: posts } = await db.execute(sql`SELECT * from posts`)
}
// MongoDB
export async function up({ payload, req }: MigrateUpArgs): Promise<void> {
const session = payload.db.sessions?.[await req.transactionID!]
const posts = await payload.db.collections.posts.collection.find({ session }).toArray()
}
```
Which was:
1. Awkward to write
2. Not documented anywhere
Now, we expose `session` and `db` to `up` and `down` functions for you:
#### MongoDB:
```ts
import { type MigrateUpArgs } from '@payloadcms/db-mongodb'
export async function up({ session, payload, req }: MigrateUpArgs): Promise<void> {
const posts = await payload.db.collections.posts.collection.find({ session }).toArray()
}
```
#### Postgres:
```ts
import { type MigrateUpArgs, sql } from '@payloadcms/db-postgres'
export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
const { rows: posts } = await db.execute(sql`SELECT * from posts`)
}
```
#### SQLite:
```ts
import { type MigrateUpArgs, sql } from '@payloadcms/db-sqlite'
export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
const { rows: posts } = await db.run(sql`SELECT * from posts`)
}
```
This actually was a thing with Postgres migrations, we already were
passing `db`, but:
1. Only for `up` and when running `payload migrate`, not for example
with `payload migrate:fresh`
2. Not documented neither in TypeScript or docs.
By ensuring we use `db`, this also fixes an issue that affects all
Postgres/SQLite migrations:
Currently, if we run `payload migration:create` with the postgres
adapter we get a file like this:
```ts
import { MigrateUpArgs, MigrateDownArgs, sql } from '@payloadcms/db-postgres'
export async function up({ payload, req }: MigrateUpArgs): Promise<void> {
await payload.db.drizzle.execute(sql`
CREATE TABLE IF NOT EXISTS "users" (
"id" serial PRIMARY KEY NOT NULL,
);
```
Looks good?
Not exactly!
`payload.db.drizzle.execute()` doesn't really use the current
transaction which can lead to some problems.
Instead, it should use the `db` from `payload.db.sessions?.[await
req.transactionID!].db` because that's where we store our Drizzle
instance with the transaction.
But now, if we generate `payload migrate:create` we get:
```ts
import { MigrateUpArgs, MigrateDownArgs, sql } from '@payloadcms/db-postgres'
export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
await db.execute(sql`
CREATE TABLE IF NOT EXISTS "users" (
"id" serial PRIMARY KEY NOT NULL,
);
```
Which is what we want, as the `db` is passed correctly here:
76428373e4/packages/drizzle/src/migrate.ts (L88-L90)
```ts
export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
const dbWithTransaction = payload.db.sessions?.[await req.transactionID!].db
payload.logger.info({ one: db === dbWithTransaction })
payload.logger.info({ two: db === payload.db.drizzle })
```
<img width="336" alt="image"
src="https://github.com/user-attachments/assets/f9fab5a9-44c2-44a9-95dd-8e5cf267f027">
Additionally, this PR refactors:
* `createMigration` with Drizzle - now we have sharable
`buildCreateMigration` in `@payloadcms/drizzle` to reduce copy-pasting
of the same logic.
* the `v2-v3` relationships migration for Postgres is now shared between
`db-postgres` and `db-vercel-postgres`, again to reduce copy-paste.
What?
Fixes issue when on parallel writes in result you can have 0 latest:
true versions.
Why?
There must be always a version with latest: true
How?
Ensures that we always have a version with latest: true by adding a
filter on createdAt < createdVersion.createdAt.
Instead, this ponentially can lead to a situation where we have 2
versions with latest: true, if they were created at the exact same time,
but this shouldn't happen in a real world scenario and it's much less
problematic than not having a version with latest: true.
Fixes https://github.com/payloadcms/payload/issues/5895
Changes from #8986
---------
Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
Ensures `sanitizeRelationshipIDs` works properly in any case
Updates predefinedMigration to work with new globals
Skips ObjectID creation errors to not fail with outdated data to the
schema.
1. Adds flag `--skip-empty` to `migrate:create` to bypass the empty
migration file prompt.
- Blank migration file will not be created if this flag is passed.
3. Adds flag `--force-accept-warning` to `migrate:fresh` to bypass the
drop database prompt
### What?
Fixes potential errors when passed to `sanitizeRelationships` `ref`
could potentially be a non object (for example `string`) because of
having in the database data in old structure.
```
"Cannot create property 'a' on string 'B'",
```
### Why?
Necessary particularly for the migration script, as it migrates
everything including versions that can have outdated data.
### How?
Ensures passed `ref` is an `object`.
### What?
Upgrades mongoose from 6 to latest `v8.8.1`
Fixes https://github.com/payloadcms/payload/issues/9171
### Why?
Compatibilty with Mongodb Atlas
### How?
- Updates deps
- Changed ObjectId from bson-objectid to use `new Type.ObjectId` from
mongoose for compatibility (only inside of db-mongodb)
- Internal type adjustments
https://github.com/payloadcms/payload/discussions/9088
BREAKING CHANGES:
All projects with existing data having versions enabled, or relationship or upload fields will want to create the predefined migration that converts all strings to ObjectIDs where needed. This can be created using `payload migrate:create --file @payloadcms/mongodb/relationships-v2-v3`.
For projects making use of the exposed Models from mongoose, review the
upgrade guides from [v6 to
v7](https://mongoosejs.com/docs/7.x/docs/migrating_to_7.html) and [v7 to
v8](https://mongoosejs.com/docs/migrating_to_8.html) and make
adjustments as needed.
---------
Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
The mongodb adapter `updateOne` method accepts an `options` argument
that allows query options to be passed to mongoose. This parameter was
added in https://github.com/payloadcms/payload/pull/8397 to support the
`upsert` operation.
This `options` parameter can also be useful when using the database
adaptor directly without going through the local api. It is true that
the Mongoose models could be used directly in such situations, but the
adapter methods include a lot of useful functionality, like for instance
the sanitization of document and relationship ids, so it is desirable to
be able to use the adapter functions while still being able to provide
mongoose query options (e.g. `{timestamps: false}`).
This PR adds the same options parameter to the other update methods of
the mongodb adapter.
### What?
Uses the `collection.dbName` property for the Mongoose model, if
defined.
### Why?
Currently, `collection.dbName` is used for the version name but not for
the actual collection name. Additionally, `autoPluralization` modifies
the `dbName` regardless. This behavior is inconsistent and contradicts
the documentation.
### How?
- Utilize `collection.dbName` instead of `collection.slug`.
- Disable `autoPluralization` for collections with a defined `dbName`.
Related: https://github.com/payloadcms/payload/discussions/9058
**BREAKING CHANGES**
If a `dbName` was previously provided, it will now be used as the
MongoDB collection name instead of the collection `slug`.
`autoPluralization` will not be applied to `dbName`.
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
This fixes a peer dependency error in our monorepo, as
eslint-plugin-jsx-a11y finally supports eslint v9.
Additionally, this officially adds TypeScript 5.6 support for
typescript-eslint.
Previously, when filtering the internal link relationship in lexical by
typing in the relationship field, it would throw an error, as that
relationship field has a relation to "date-fields".
Currently, Payload renders all custom components on initial compile of
the admin panel. This is problematic for two key reasons:
1. Custom components do not receive contextual data, i.e. fields do not
receive their field data, edit views do not receive their document data,
etc.
2. Components are unnecessarily rendered before they are used
This was initially required to support React Server Components within
the Payload Admin Panel for two key reasons:
1. Fields can be dynamically rendered within arrays, blocks, etc.
2. Documents can be recursively rendered within a "drawer" UI, i.e.
relationship fields
3. Payload supports server/client component composition
In order to achieve this, components need to be rendered on the server
and passed as "slots" to the client. Currently, the pattern for this is
to render custom server components in the "client config". Then when a
view or field is needed to be rendered, we first check the client config
for a "pre-rendered" component, otherwise render our client-side
fallback component.
But for the reasons listed above, this pattern doesn't exactly make
custom server components very useful within the Payload Admin Panel,
which is where this PR comes in. Now, instead of pre-rendering all
components on initial compile, we're able to render custom components
_on demand_, only as they are needed.
To achieve this, we've established [this
pattern](https://github.com/payloadcms/payload/pull/8481) of React
Server Functions in the Payload Admin Panel. With Server Functions, we
can iterate the Payload Config and return JSX through React's
`text/x-component` content-type. This means we're able to pass
contextual props to custom components, such as data for fields and
views.
## Breaking Changes
1. Add the following to your root layout file, typically located at
`(app)/(payload)/layout.tsx`:
```diff
/* THIS FILE WAS GENERATED AUTOMATICALLY BY PAYLOAD. */
/* DO NOT MODIFY IT BECAUSE IT COULD BE REWRITTEN AT ANY TIME. */
+ import type { ServerFunctionClient } from 'payload'
import config from '@payload-config'
import { RootLayout } from '@payloadcms/next/layouts'
import { handleServerFunctions } from '@payloadcms/next/utilities'
import React from 'react'
import { importMap } from './admin/importMap.js'
import './custom.scss'
type Args = {
children: React.ReactNode
}
+ const serverFunctions: ServerFunctionClient = async function (args) {
+ 'use server'
+ return handleServerFunctions({
+ ...args,
+ config,
+ importMap,
+ })
+ }
const Layout = ({ children }: Args) => (
<RootLayout
config={config}
importMap={importMap}
+ serverFunctions={serverFunctions}
>
{children}
</RootLayout>
)
export default Layout
```
2. If you were previously posting to the `/api/form-state` endpoint, it
no longer exists. Instead, you'll need to invoke the `form-state` Server
Function, which can be done through the _new_ `getFormState` utility:
```diff
- import { getFormState } from '@payloadcms/ui'
- const { state } = await getFormState({
- apiRoute: '',
- body: {
- // ...
- },
- serverURL: ''
- })
+ const { getFormState } = useServerFunctions()
+
+ const { state } = await getFormState({
+ // ...
+ })
```
## Breaking Changes
```diff
- useFieldProps()
- useCellProps()
```
More details coming soon.
---------
Co-authored-by: Alessio Gravili <alessio@gravili.de>
Co-authored-by: Jarrod Flesch <jarrodmflesch@gmail.com>
Co-authored-by: James <james@trbl.design>
### What?
Adds full support for the point field to Postgres and Vercel Postgres
adapters through the Postgis extension. Fully the same API as with
MongoDB, including support for `near`, `within` and `intersects`
operators.
Additionally, exposes to adapter args:
*
`tablesFilter`https://orm.drizzle.team/docs/drizzle-kit-push#including-tables-schemas-and-extensions.
* `extensions` list of extensions to create, for example `['vector',
'pg_search']`, `postgis` is created automatically if there's any point
field
### Why?
It's essential to support that field type, especially if the postgres
adapter should be out of beta on 3.0 stable.
### How?
* Bumps `drizzle-orm` to `0.36.1` and `drizzle-kit` to `0.28.0` as we
need this change https://github.com/drizzle-team/drizzle-orm/pull/3141
* Uses its functions to achieve querying functionality, for example the
`near` operator works through `ST_DWithin` or `intersects` through
`ST_Intersects`.
* Removes MongoDB condition from all point field tests, but keeps for
SQLite
Resolves these discussions:
https://github.com/payloadcms/payload/discussions/8996https://github.com/payloadcms/payload/discussions/8644
### What?
Moved the logic for copying the data.id to data._id to the mongoose
adapter.
### Why?
If you have any hooks that need to set the `id`, the value does not get
sent to mongodb as you would expect since it was copied before the
beforeValidate hooks.
### How?
Now data._id is assigned only in the mongodb adapter's `create`
function.
BREAKING CHANGES:
When using custom ID fields, if you have any collection hooks for
beforeValidate, beforeChange then `data._id` will no longer be assigned
as this happens now in the database adapter. Use `data.id` instead.
### What?
Fixes issue with incorrect `totalDocs` value when an aggregation is used
for `find`.
Previously, `limit: 5` for example always returned `totalDocs: 5`.
### Why?
`totalDocs` must be returned correctly.
### How?
Removes `$limit` from the pipeline, as `Model.aggregatePaginate` handles
it by itself.
### What?
Makes it possible to filter join documents using a `where` added
directly in the config.
### Why?
It makes the join field more powerful for adding contextual meaning to
the documents being returned. For example, maybe you have a
`requiresAction` field that you set and you can have a join that
automatically filters the documents to those that need attention.
### How?
In the database adapter, we merge the requested `where` to the `where`
defined on the field.
On the frontend the results are filtered using the `filterOptions`
property in the component.
Fixes
https://github.com/payloadcms/payload/discussions/8936https://github.com/payloadcms/payload/discussions/8937
---------
Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
### What?
Fixes `select` handling for properties inside of unnamed tabs using the
mongodb adapter.
Additionally, refactors `traverseFields` in drizzle to reuse logic from
groups / collapsible or rows if unnamed.
### Why?
`select` must work for any fields.
### How?
Fixes the `'tab'` case in `buildProjectionFromSelect` to handle when the
field is an unnamed tab.
Adds extra tests for named tabs / unnamed.
### What?
Properly specifies `$lookup.from` when the collection name is singular.
### Why?
MongoDB can pluralize the collection name and so can be different for
singular ones.
### How?
Uses the collection name from the driver directly
`adapter.collections[slug].collection.name` instead of just `slug`.
Adds `select` which is used to specify the field projection for local
and rest API calls. This is available as an optimization to reduce the
payload's of requests and make the database queries more efficient.
Includes:
- [x] generate types for the `select` property
- [x] infer the return type by `select` with 2 modes - include (`field:
true`) and exclude (`field: false`)
- [x] lots of integration tests, including deep fields / localization
etc
- [x] implement the property in db adapters
- [x] implement the property in the local api for most operations
- [x] implement the property in the rest api
- [x] docs
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
### What?
Fixes the issue with `in` querying when the collection has a join field.
### Why?
When using `.aggregate`, MongoDB doesn't cast a comma delimited value
for the `$in` operator to an array automatically as it's not handled by
Mongoose.
### How?
Sanitizes the incoming value to an array if it should.
Fixes https://github.com/payloadcms/payload/issues/8901
### What?
Allow specifying the defaultSort and defaultLimit to use for populating
a join field
### Why?
It is much easier to set defaults rather than be forced to always call
the join query using the query pattern ("?joins[categories][limit]=0").
### How?
See docs and type changes
This change adds support for sort with multiple fields in local API and
REST API. Related discussion #2089
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
This PR makes a more clear gap between `version_createdAt` /
`version_updatedAt` and `createdAt` / `updatedAt` columns / fields in
mongodb.
- `createdAt` - This should be a new value in a new version. Before this
change it was the same all the time. Should remain the same on autosave.
- The same for `updatedAt`, but it should be updated on every change
(including autosave)
- `version_createdAt` - Should remain equal to `createdAt` from the
parent collection / table
- `version_updatedAt` - On a latest version it makes sense this be the
same as `updatedAt` from the parent collection / table, as all the
`version_*` fields should be just synced with it
