This PR adds a feature which fixes another issue with migrations in
Postgres and does few refactors that significantly reduce code
duplication.
Previously, if you needed to use the underlying database directly in
migrations with the active transaction (for example to execute raw SQL),
created from `payload create:migration`, as `req` doesn't work there you
had to do something like this:
```ts
// Postgres
export async function up({ payload, req }: MigrateUpArgs): Promise<void> {
const db = payload.db.sessions?.[await req.transactionID!].db ?? payload.db
const { rows: posts } = await db.execute(sql`SELECT * from posts`)
}
// MongoDB
export async function up({ payload, req }: MigrateUpArgs): Promise<void> {
const session = payload.db.sessions?.[await req.transactionID!]
const posts = await payload.db.collections.posts.collection.find({ session }).toArray()
}
```
Which was:
1. Awkward to write
2. Not documented anywhere
Now, we expose `session` and `db` to `up` and `down` functions for you:
#### MongoDB:
```ts
import { type MigrateUpArgs } from '@payloadcms/db-mongodb'
export async function up({ session, payload, req }: MigrateUpArgs): Promise<void> {
const posts = await payload.db.collections.posts.collection.find({ session }).toArray()
}
```
#### Postgres:
```ts
import { type MigrateUpArgs, sql } from '@payloadcms/db-postgres'
export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
const { rows: posts } = await db.execute(sql`SELECT * from posts`)
}
```
#### SQLite:
```ts
import { type MigrateUpArgs, sql } from '@payloadcms/db-sqlite'
export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
const { rows: posts } = await db.run(sql`SELECT * from posts`)
}
```
This actually was a thing with Postgres migrations, we already were
passing `db`, but:
1. Only for `up` and when running `payload migrate`, not for example
with `payload migrate:fresh`
2. Not documented neither in TypeScript or docs.
By ensuring we use `db`, this also fixes an issue that affects all
Postgres/SQLite migrations:
Currently, if we run `payload migration:create` with the postgres
adapter we get a file like this:
```ts
import { MigrateUpArgs, MigrateDownArgs, sql } from '@payloadcms/db-postgres'
export async function up({ payload, req }: MigrateUpArgs): Promise<void> {
await payload.db.drizzle.execute(sql`
CREATE TABLE IF NOT EXISTS "users" (
"id" serial PRIMARY KEY NOT NULL,
);
```
Looks good?
Not exactly!
`payload.db.drizzle.execute()` doesn't really use the current
transaction which can lead to some problems.
Instead, it should use the `db` from `payload.db.sessions?.[await
req.transactionID!].db` because that's where we store our Drizzle
instance with the transaction.
But now, if we generate `payload migrate:create` we get:
```ts
import { MigrateUpArgs, MigrateDownArgs, sql } from '@payloadcms/db-postgres'
export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
await db.execute(sql`
CREATE TABLE IF NOT EXISTS "users" (
"id" serial PRIMARY KEY NOT NULL,
);
```
Which is what we want, as the `db` is passed correctly here:
76428373e4/packages/drizzle/src/migrate.ts (L88-L90)
```ts
export async function up({ db, payload, req }: MigrateUpArgs): Promise<void> {
const dbWithTransaction = payload.db.sessions?.[await req.transactionID!].db
payload.logger.info({ one: db === dbWithTransaction })
payload.logger.info({ two: db === payload.db.drizzle })
```
<img width="336" alt="image"
src="https://github.com/user-attachments/assets/f9fab5a9-44c2-44a9-95dd-8e5cf267f027">
Additionally, this PR refactors:
* `createMigration` with Drizzle - now we have sharable
`buildCreateMigration` in `@payloadcms/drizzle` to reduce copy-pasting
of the same logic.
* the `v2-v3` relationships migration for Postgres is now shared between
`db-postgres` and `db-vercel-postgres`, again to reduce copy-paste.
This fixes a peer dependency error in our monorepo, as
eslint-plugin-jsx-a11y finally supports eslint v9.
Additionally, this officially adds TypeScript 5.6 support for
typescript-eslint.
## Description
1. Adds ability to publish a specific individual locale (collections and
globals)
2. Shows published locale in versions list and version comparison
3. Adds new int tests to `versions` test suite
- [X] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
## Type of change
- [X] New feature (non-breaking change which adds functionality)
- [ ] This change requires a documentation update
## Checklist:
- [X] I have added tests that prove my fix is effective or that my
feature works
- [X] Existing test suite passes locally with my changes
- [ ] I have made corresponding changes to the documentation
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
- Abstract shared sql code to a new drizzle package
- Adds sqlite package, not ready to publish until drizzle patches some
issues
- Add `transactionOptions` to allow customizing or disabling db
transactions
- Adds "experimental" label to the `schemaName` property until drizzle
patches an issue
## Description
This is the beta (v3) PR for the v2 PR
[here](https://github.com/payloadcms/payload/pull/6857)
Addresses #6800, #5108
- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## Checklist:
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
Removes PayloadRequestWithData in favour of just PayloadRequest with
optional types for `data` and `locale`
`addDataAndFileToRequest` and `addLocalesToRequestFromData` now takes in
a single argument instead of an object
```ts
// before
await addDataAndFileToRequest({ request: req })
addLocalesToRequestFromData({ request: req })
// current
await addDataAndFileToRequest(req)
addLocalesToRequestFromData(req)
```
---------
Co-authored-by: Paul Popus <paul@nouance.io>
**BREAKING:**
- Type narrowing for `relationTo` props on filterOptions, relationship
fields and upload fields
- Type narrowing for arguments of lexical relationship, link and upload
features
# Breaking Changes
### New file import locations
Exports from the `payload` package have been _significantly_ cleaned up.
Now, just about everything is able to be imported from `payload`
directly, rather than an assortment of subpath exports. This means that
things like `import { buildConfig } from 'payload/config'` are now just
imported via `import { buildConfig } from 'payload'`. The mental model
is significantly simpler for developers, but you might need to update
some of your imports.
Payload now exposes only three exports:
1. `payload` - all types and server-only Payload code
2. `payload/shared` - utilities that can be used in either the browser
or in Node environments
3. `payload/node` - heavy utilities that should only be imported in Node
scripts and never be imported into bundled code like Next.js
### UI library pre-bundling
With this release, we've dramatically sped up the compile time for
Payload by pre-bundling our entire UI package for use inside of the
Payload admin itself. There are new exports that should be used within
Payload custom components:
1. `@payloadcms/ui/client` - all client components
2. `@payloadcms/ui/server` - all server components
For all of your custom Payload admin UI components, you should be
importing from one of these two pre-compiled barrel files rather than
importing from the more deeply nested exports directly. That will keep
compile times nice and speedy, and will also make sure that the bundled
JS for your admin UI is kept small.
For example, whereas before, if you imported the Payload `Button`, you
would have imported it like this:
```ts
import { Button } from '@payloadcms/ui/elements/Button'
```
Now, you would import it like this:
```ts
import { Button } from '@payloadcms/ui/client'
```
This is a significant DX / performance optimization that we're pretty
pumped about.
However, if you are importing or re-using Payload UI components
_outside_ of the Payload admin UI, for example in your own frontend
apps, you can import from the individual component exports which will
make sure that the bundled JS is kept to a minimum in your frontend
apps. So in your own frontend, you can continue to import directly to
the components that you want to consume rather than importing from the
pre-compiled barrel files.
Individual component exports will now come with their corresponding CSS
and everything will work perfectly as-expected.
### Specific exports have changed
- `'@payloadcms/ui/templates/Default'` and
`'@payloadcms/ui/templates/Minimal`' are now exported from
`'@payloadcms/next/templates'`
- Old: `import { LogOut } from '@payloadcms/ui/icons/LogOut'` new:
`import { LogOutIcon } from '@payloadcms/ui/icons/LogOut'`
## Background info
In effort to make local dev as fast as possible, we need to import as
few files as possible so that the compiler has less to process. One way
we've achieved this in the Admin Panel was to _remove_ all .scss imports
from all components in the `@payloadcms/ui` module using a build
process. This stripped all `import './index.scss'` statements out of
each component before injecting them into `dist`. Instead, it bundles
all of the CSS into a single `main.css` file, and we import _that_ at
the root of the app.
While this concept is _still_ the right solution to the problem, this
particular approach is not viable when using these components outside
the Admin Panel, where not only does this root stylesheet not exist, but
where it would also bloat your app with unused styles. Instead, we need
to _keep_ these .scss imports in place so they are imported directly
alongside your components, as expected. Then, we need create a _new_
build step that _separately_ compiles the components _without_ their
stylesheets—this way your app can consume either as needed from the new
`client` and `server` barrel files within `@payloadcms/ui`, i.e. from
within `@payloadcms/next` and all other admin-specific packages and
plugins.
This way, all other applications will simply import using the direct
file paths, just as they did before. Except now they come with
stylesheets.
And we've gotten a pretty awesome initial compilation performance boost.
---------
Co-authored-by: James <james@trbl.design>
Co-authored-by: Alessio Gravili <alessio@gravili.de>
## Description
fixes#6630
# BREAKING CHANGES
This only applies to you if you using db-postgres and have created the
`v2-v3-relationships` migration released in
[v3.0.0-beta.39](https://github.com/payloadcms/payload/releases/tag/v3.0.0-beta.39)
from @payloadcms/db-postgres <= v3.0.0-beta.40.
### Steps to fix
- Delete the existing v2-v3-relationships migration file.
- If changes were made to your config since the previous migration was
made, you will need to revert those by checking out a previous commit in
your version control.
- Recreate the migration using `payload migrate:create --file
@payloadcms/db-postgres/relationships-v2-v3` to make the migration with
the snapshot .json file.
BREAKING CHANGE:
Moves `upload` field and `relationship` fields with `hasMany: false` &
`relationTo: string` from the many-to-many `_rels` join table to simple
columns. This only affects Postgres database users.
## TL;DR
We have dramatically simplified the storage of simple relationships in
relational databases to boost performance and align with more expected
relational paradigms. If you are using the beta Postgres adapter, and
you need to keep simple relationship data, you'll need to run a
migration script that we provide you.
### Background
For example, prior to this update, a collection of "posts" with a simple
`hasMany: false` and `relationTo: 'categories'` field would have a
`posts_rels` table where the category relations would be stored.
This was somewhat unnecessary as simple relations like this can be
expressed with a `category_id` column which is configured as a foreign
key. This also introduced added complexity for dealing directly with the
database if all you have are simple relations.
### Who needs to migrate
You need to migrate if you are using the beta Postgres database adapter
and any of the following applies to you.
- If you have versions enabled on any collection / global
- If you use the `upload` field
- If you have relationship fields that are `hasMany: false` (default)
and `relationTo` to a single category ([has
one](https://payloadcms.com/docs/fields/relationship#has-one)) relations
### We have a migration for you
Even though the Postgres adapter is in beta, we've prepared a predefined
migration that will work out of the box for you to migrate from an
earlier version of the adapter to the most recent version easily.
It makes the schema changes in step with actually moving the data from
the old locations to the new before adding any null constraints and
dropping the old columns and tables.
### How to migrate
The steps to preserve your data while making this update are as follows.
These steps are the same whether you are moving from Payload v2 to v3 or
a previous version of v3 beta to the most recent v3 beta.
**Important: during these steps, don't start the dev server unless you
have `push: false` set on your Postgres adapter.**
#### Step 1 - backup
Always back up your database before performing big changes, especially
in production cases.
#### Step 2 - create a pre-update migration
Before updating to new Payload and Postgres adapter versions, run
`payload migrate:create` without any other config changes to have a
prior snapshot of the schema from the previous adapter version
#### Step 3 - if you're migrating a dev DB, delete the dev `push` row
from your `payload_migrations` table
If you're migrating a dev database where you have the default setting to
push database changes directly to your DB, and you need to preserve data
in your development database, then you need to delete a `dev` migration
record from your database.
Connect directly to your database in any tool you'd like and delete the
dev push record from the `payload_migrations` table using the following
SQL statement:
```sql
DELETE FROM payload_migrations where batch = -1`
```
#### Step 4 - update Payload and Postgres versions to most recent
Update packages, making sure you have matching versions across all
`@payloadcms/*` and `payload` packages (including
`@payloadcms/db-postgres`)
#### Step 5 - create the predefined migration
Run the following command to create the predefined migration we've
provided:
```
payload migrate:create --file @payloadcms/db-postgres/relationships-v2-v3
```
#### Step 6 - migrate!
Run migrations with the following command:
```
payload migrate
```
Assuming the migration worked, you can proceed to commit this change and
distribute it to be run on all other environments.
Note that if two servers connect to the same database, only one should
be running migrations to avoid transaction conflicts.
Related discussion:
https://github.com/payloadcms/payload/discussions/4163
---------
Co-authored-by: James <james@trbl.design>
Co-authored-by: PatrikKozak <patrik@payloadcms.com>
