Previously we had been downgrading rimraf to v3 simply to handle clean
with glob patterns across platforms. In v4 and newer of rimraf you can
add `-g` to use glob patterns.
This change updates rimraf and adds the flag to handle globs in our
package scripts to be windows compatible.
### What?
The join field had a limitation imposed that prevents it from targeting
polymorphic relationship fields. With this change we can support any
relationship fields.
### Why?
Improves the functionality of join field.
### How?
Extended the database adapters and removed the config sanitization that
would throw an error when polymorphic relationships were used.
Fixes #
Previously, queries like this didn't work:
```ts
const res = await payload.find({
collection: 'polymorphic-relationships',
where: {
polymorphicLocalized: {
equals: {
relationTo: 'movies',
value: movie.id,
},
},
},
})
```
This was due to the incorrectly passed path to MongoDB without
`.{locale}` suffix.
Additionally, to MongoDB now we send:
```
{
$or: [
{
polymorphic: {
$eq: {
relationTo: formattedValue.relationTo,
value: formattedValue.value,
},
},
},
{
polymorphic: {
$eq: {
relationTo: 'movies',
value: 'some-id',
},
},
},
],
},
```
Instead of:
```
{
$and: [
{
'polymorphic.relationTo': {
$eq: 'movies ',
},
},
{
'polymorphic.value': {
$eq: 'some-id ',
},
},
],
}
```
To match the _exact_ value. This is essential when we do querying by
relationships with `hasMany: true` and custom IDs that can be repeated.
`$or` is needed if for some reason keys are stored in the DB in a
different order
### What?
Previously, the `req` argument:
In database operations (e.g `payload.db`) was required and you needed to
pass the whole `req` with all the properties. This is confusing because
in database operations we never use its properties outside of
`req.transactionID` and `req.t`, both of which should be optional as
well.
Now, you don't have to do that cast:
```ts
payload.db.findOne({
collection: 'posts',
req: {} as PayloadRequest,
where: {
id: {
equals: 1,
},
},
})
```
Becomes:
```ts
payload.db.findOne({
collection: 'posts',
where: {
id: {
equals: 1,
},
},
})
```
If you need to use transactions, you're not required to do the `as` cast
as well now, as the `req` not only optional but also partial -
`Partial<PayloadRequest>`.
`initTransaction`, `commitTransaction`, `killTransaction` utilities are
typed better now as well. They do not require to you pass all the
properties of `req`, but only `payload` -
`MarkRequired<Partial<PayloadRequest>, 'payload'>`
```ts
const req = { payload }
await initTransaction(req)
await payload.db.create({
collection: "posts",
data: {},
req
})
await commitTransaction(req)
```
The same for the Local API. Internal operations (for example
`packages/payload/src/collections/operations/find.ts`) still accept the
whole `req`, but local ones
(`packages/payload/src/collections/operations/local/find.ts`) which are
used through `payload.` now accept `Partial<PayloadRequest>`, as they
pass it through to internal operations with `createLocalReq`.
So now, this is also valid, while previously you had to do `as` cast for
`req`.
```ts
const req = { payload }
await initTransaction(req)
await payload.create({
collection: "posts",
data: {},
req
})
await commitTransaction(req)
```
Marked as deprecated `PayloadRequest['transactionIDPromise']` to remove
in the next major version. It was never used anywhere.
Refactored `withSession` that returns an object to `getSession` that
returns just `ClientSession`. Better type safety for arguments
Deduplicated in all drizzle operations to `getTransaction(this, req)`
utility:
```ts
const db = this.sessions[await req?.transactionID]?.db || this.drizzle
```
Added fallback for throwing unique validation errors in database
operations when `req.t` is not available.
In migration `up` and `down` functions our `req` is not partial, while
we used to passed `req` with only 2 properties - `payload` and
`transactionID`. This is misleading and you can't access for example
`req.t`.
Now, to achieve "real" full `req` - we generate it with `createLocalReq`
in all migration functions.
This all is backwards compatible. In all public API places where you
expect the full `req` (like hooks) you still have it.
### Why?
Better DX, more expected types, less errors because of types casting.
Adds the ability to pass additional schema options for collections with:
```ts
mongooseAdapter({
collectionsSchemaOptions: {
posts: {
strict: false,
},
},
})
```
This changes relates to these:
- https://github.com/payloadcms/payload/issues/4533
- https://github.com/payloadcms/payload/discussions/4534
It is a proposal to set custom schema options for mongoose driver.
I understand this got introduced into `main` v2 after `beta` branch was
created so this feature got lost.
- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
<!--
Thank you for the PR! Please go through the checklist below and make
sure you've completed all the steps.
Please review the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository if you haven't already.
The following items will ensure that your PR is handled as smoothly as
possible:
- PR Title must follow conventional commits format. For example, `feat:
my new feature`, `fix(plugin-seo): my fix`.
- Minimal description explained as if explained to someone not
immediately familiar with the code.
- Provide before/after screenshots or code diffs if applicable.
- Link any related issues/discussions from GitHub or Discord.
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Fixes #
-->
---------
Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
Should fix messed up import suggestions and simplifies all tsconfigs
through inheritance.
One main issue was that packages were inheriting `baseURL: "."` from the
root tsconfig. This caused incorrect import suggestions that start with
"packages/...".
This PR ensures that packages do not inherit this baseURL: "." property,
while ensuring the root, non-inherited tsconfig still keeps it to get
tests to work (the importMap needs it)
This PR adds a feature which fixes another issue with migrations in
Postgres and does few refactors that significantly reduce code
duplication.
Previously, if you needed to use the underlying database directly in
migrations with the active transaction (for example to execute raw SQL),
created from `payload create:migration`, as `req` doesn't work there you
had to do something like this:
```ts
// Postgres
export async function up({ payload, req }: MigrateUpArgs): Promise<void> {
const db = payload.db.sessions?.[await req.transactionID!].db ?? payload.db
const { rows: posts } = await db.execute(sql`SELECT * from posts`)
}
// MongoDB
export async function up({ payload, req }: MigrateUpArgs): Promise<void> {
const session = payload.db.sessions?.[await req.transactionID!]
const posts = await payload.db.collections.posts.collection.find({ session }).toArray()
}
```
Which was:
1. Awkward to write
2. Not documented anywhere
Now, we expose `session` and `db` to `up` and `down` functions for you:
#### MongoDB:
```ts
import { type MigrateUpArgs } from '@payloadcms/db-mongodb'
export async function up({ session, payload, req }: MigrateUpArgs): Promise<void> {
const posts = await payload.db.collections.posts.collection.find({ session }).toArray()
}
```
#### Postgres:
```ts
import { type MigrateUpArgs, sql } from '@payloadcms/db-postgres'
export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
const { rows: posts } = await db.execute(sql`SELECT * from posts`)
}
```
#### SQLite:
```ts
import { type MigrateUpArgs, sql } from '@payloadcms/db-sqlite'
export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
const { rows: posts } = await db.run(sql`SELECT * from posts`)
}
```
This actually was a thing with Postgres migrations, we already were
passing `db`, but:
1. Only for `up` and when running `payload migrate`, not for example
with `payload migrate:fresh`
2. Not documented neither in TypeScript or docs.
By ensuring we use `db`, this also fixes an issue that affects all
Postgres/SQLite migrations:
Currently, if we run `payload migration:create` with the postgres
adapter we get a file like this:
```ts
import { MigrateUpArgs, MigrateDownArgs, sql } from '@payloadcms/db-postgres'
export async function up({ payload, req }: MigrateUpArgs): Promise<void> {
await payload.db.drizzle.execute(sql`
CREATE TABLE IF NOT EXISTS "users" (
"id" serial PRIMARY KEY NOT NULL,
);
```
Looks good?
Not exactly!
`payload.db.drizzle.execute()` doesn't really use the current
transaction which can lead to some problems.
Instead, it should use the `db` from `payload.db.sessions?.[await
req.transactionID!].db` because that's where we store our Drizzle
instance with the transaction.
But now, if we generate `payload migrate:create` we get:
```ts
import { MigrateUpArgs, MigrateDownArgs, sql } from '@payloadcms/db-postgres'
export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
await db.execute(sql`
CREATE TABLE IF NOT EXISTS "users" (
"id" serial PRIMARY KEY NOT NULL,
);
```
Which is what we want, as the `db` is passed correctly here:
76428373e4/packages/drizzle/src/migrate.ts (L88-L90)
```ts
export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
const dbWithTransaction = payload.db.sessions?.[await req.transactionID!].db
payload.logger.info({ one: db === dbWithTransaction })
payload.logger.info({ two: db === payload.db.drizzle })
```
<img width="336" alt="image"
src="https://github.com/user-attachments/assets/f9fab5a9-44c2-44a9-95dd-8e5cf267f027">
Additionally, this PR refactors:
* `createMigration` with Drizzle - now we have sharable
`buildCreateMigration` in `@payloadcms/drizzle` to reduce copy-pasting
of the same logic.
* the `v2-v3` relationships migration for Postgres is now shared between
`db-postgres` and `db-vercel-postgres`, again to reduce copy-paste.
What?
Fixes issue when on parallel writes in result you can have 0 latest:
true versions.
Why?
There must be always a version with latest: true
How?
Ensures that we always have a version with latest: true by adding a
filter on createdAt < createdVersion.createdAt.
Instead, this ponentially can lead to a situation where we have 2
versions with latest: true, if they were created at the exact same time,
but this shouldn't happen in a real world scenario and it's much less
problematic than not having a version with latest: true.
Fixes https://github.com/payloadcms/payload/issues/5895
Changes from #8986
---------
Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
Ensures `sanitizeRelationshipIDs` works properly in any case
Updates predefinedMigration to work with new globals
Skips ObjectID creation errors to not fail with outdated data to the
schema.
1. Adds flag `--skip-empty` to `migrate:create` to bypass the empty
migration file prompt.
- Blank migration file will not be created if this flag is passed.
3. Adds flag `--force-accept-warning` to `migrate:fresh` to bypass the
drop database prompt
### What?
Fixes potential errors when passed to `sanitizeRelationships` `ref`
could potentially be a non object (for example `string`) because of
having in the database data in old structure.
```
"Cannot create property 'a' on string 'B'",
```
### Why?
Necessary particularly for the migration script, as it migrates
everything including versions that can have outdated data.
### How?
Ensures passed `ref` is an `object`.
### What?
Upgrades mongoose from 6 to latest `v8.8.1`
Fixes https://github.com/payloadcms/payload/issues/9171
### Why?
Compatibilty with Mongodb Atlas
### How?
- Updates deps
- Changed ObjectId from bson-objectid to use `new Type.ObjectId` from
mongoose for compatibility (only inside of db-mongodb)
- Internal type adjustments
https://github.com/payloadcms/payload/discussions/9088
BREAKING CHANGES:
All projects with existing data having versions enabled, or relationship or upload fields will want to create the predefined migration that converts all strings to ObjectIDs where needed. This can be created using `payload migrate:create --file @payloadcms/mongodb/relationships-v2-v3`.
For projects making use of the exposed Models from mongoose, review the
upgrade guides from [v6 to
v7](https://mongoosejs.com/docs/7.x/docs/migrating_to_7.html) and [v7 to
v8](https://mongoosejs.com/docs/migrating_to_8.html) and make
adjustments as needed.
---------
Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
The mongodb adapter `updateOne` method accepts an `options` argument
that allows query options to be passed to mongoose. This parameter was
added in https://github.com/payloadcms/payload/pull/8397 to support the
`upsert` operation.
This `options` parameter can also be useful when using the database
adaptor directly without going through the local api. It is true that
the Mongoose models could be used directly in such situations, but the
adapter methods include a lot of useful functionality, like for instance
the sanitization of document and relationship ids, so it is desirable to
be able to use the adapter functions while still being able to provide
mongoose query options (e.g. `{timestamps: false}`).
This PR adds the same options parameter to the other update methods of
the mongodb adapter.
