Commit Graph

323 Commits

Author SHA1 Message Date
Elliot DeNolf
2835e1d709 feat: abstract postgres base adapter (#7732)
Abstracts Postgres base adapter in order to allow future postgres-based
adapters.
2024-08-16 18:51:39 -04:00
Elliot DeNolf
5fc9f76406 feat: filename compound index (#7651)
Allow a compound index to be used for upload collections via a
`filenameCompoundIndex` field. Previously, `filename` was always treated
as unique.

Usage:

```ts
{
  slug: 'upload-field',
   upload: {
     // Slugs to include in compound index
     filenameCompoundIndex: ['filename', 'alt'],
  },
}
```
2024-08-13 13:55:10 -04:00
Elliot DeNolf
fa3d250053 feat: indent migration sql (#7475)
Properly indent migration sql
2024-08-09 22:41:28 -04:00
James Mikrut
1cb1e5e8b3 feat(db-*): allows for running migrations in production automatically (#7563)
## Description

Introduces a pattern for running migrations upon Payload init in
production.
2024-08-07 13:57:12 -04:00
Dan Ribbens
46f70d9df4 fix(db-postgres): #7492 migrate snapshots (#7540)
## Description

Fixes #7492 

In order to run createMigration, we need to read in the previous
snapshot file if one exists. When that snapshot was generated from an
older version of drizzle-kit, we have to first migrate it up match the
latest version for drizzle to generate the new migration. This change
adds in the call to check the version and migrate the snapshot if
needed.
2024-08-07 13:49:08 -04:00
Elliot DeNolf
95fcd13929 fix(db-*): drizzle enums, bump drizzle-kit (#7514)
- bumps drizzle-kit
- Fixes https://github.com/payloadcms/payload/issues/7492 Enum issue.
2024-08-05 14:53:21 -04:00
Dan Ribbens
449c16d28f fix(db-postgres): incorrect schema type on adapter (#7459)
fixes a db-postgres type issue that was introduced in
https://github.com/payloadcms/payload/pull/7453
2024-08-01 10:39:54 -04:00
Dan Ribbens
075819964d fix(db-postgres, db-sqlite): enum schema (#7453)
- updates drizzle-kit and drizzle-orm
- fix enum creation to fully support custom schemas
- sqlite by default will not use transactions
2024-07-31 16:42:00 -04:00
Dan Ribbens
695ef32d1e feat(db-*): add defaultValues to database schemas (#7368)
## Description

Prior to this change, the `defaultValue` for fields have only been used
in the application layer of Payload. With this change, you get the added
benefit of having the database columns get the default also. This is
especially helpful when adding new columns to postgres with existing
data to avoid needing to write complex migrations. In MongoDB this
change applies the default to the Mongoose model which is useful when
calling payload.db.create() directly.

This only works for statically defined values.

🙏 A big thanks to @r1tsuu for the feature and implementation idea as I
lifted some code from PR https://github.com/payloadcms/payload/pull/6983

- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.

## Type of change

- [x] New feature (non-breaking change which adds functionality)
- [x] This change requires a documentation update

## Checklist:

- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
- [x] I have made corresponding changes to the documentation
2024-07-30 13:41:18 -04:00
Dan Ribbens
354588898f feat(db-*, payload): better transactions (#7395)
## Description

### payload
- Removes calls to beginTransaction and commitTransaction from read
operations

### db-sqlite, db-postgres
- beginTransaction() options are passed through and used to create a
transaction
- declare module type adds beginTransaction with proper transaction
config args for postgres and sqlite
2024-07-29 15:35:19 -04:00
Dan Ribbens
1dc428823a fix(db-postgres): migration template type error (#7403)
Fixes #7402

This fixes a regression from changes to the postgres migration template
that were incorrect. It also fixes other type errors for
`payload.db.drizzle` which needed to be declared for postgres to avoid
confusing it with Libsql for SQLite.
2024-07-27 21:52:36 -04:00
Elliot DeNolf
55c6ce92b0 fix(db-postgres): properly reference drizzle createTableName function (#7383)
Fix incorrect relative import of drizzle package's `createTableName`
function. Now uses proper package import.

Fixes #7373
2024-07-26 10:08:31 -04:00
Dan Ribbens
09ad6e4280 feat(drizzle): abstract shared sql code to new package (#7320)
- Abstract shared sql code to a new drizzle package
- Adds sqlite package, not ready to publish until drizzle patches some
issues
- Add `transactionOptions` to allow customizing or disabling db
transactions
- Adds "experimental" label to the `schemaName` property until drizzle
patches an issue
2024-07-24 12:43:29 -04:00
Patrik
fb72d19d6c fix: graphql query concurrency issues (#6925)
## Description

This is the beta (v3) PR for the v2 PR
[here](https://github.com/payloadcms/payload/pull/6857)

Addresses #6800, #5108

- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## Checklist:

- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
2024-07-08 15:55:04 +00:00
Jarrod Flesch
0711f880ff chore!: simplify api handler (#6910)
Removes PayloadRequestWithData in favour of just PayloadRequest with
optional types for `data` and `locale`

`addDataAndFileToRequest` and `addLocalesToRequestFromData` now takes in
a single argument instead of an object

```ts
// before
await addDataAndFileToRequest({ request: req })
addLocalesToRequestFromData({ request: req })

// current
await addDataAndFileToRequest(req)
addLocalesToRequestFromData(req)
```

---------

Co-authored-by: Paul Popus <paul@nouance.io>
2024-07-02 09:47:03 -04:00
Alessio Gravili
8f346dfb62 feat!: show detailed validation errors in console (#6551)
BREAKING: `ValidationError` now requires the `global` or `collection`
slug, as well as an `errors` property. The actual errors are no longer
at the top-level.
2024-06-28 16:35:35 -04:00
Alessio Gravili
ccbaee43cc feat!: various type improvements (#6385)
**BREAKING:**
- Type narrowing for `relationTo` props on filterOptions, relationship
fields and upload fields
- Type narrowing for arguments of lexical relationship, link and upload
features
2024-06-24 16:38:46 -04:00
Jacob Fletcher
9e76c8f4e3 feat!: prebundle payload, ui, richtext-lexical (#6579)
# Breaking Changes

### New file import locations

Exports from the `payload` package have been _significantly_ cleaned up.
Now, just about everything is able to be imported from `payload`
directly, rather than an assortment of subpath exports. This means that
things like `import { buildConfig } from 'payload/config'` are now just
imported via `import { buildConfig } from 'payload'`. The mental model
is significantly simpler for developers, but you might need to update
some of your imports.

Payload now exposes only three exports:

1. `payload` - all types and server-only Payload code
2. `payload/shared` - utilities that can be used in either the browser
or in Node environments
3. `payload/node` - heavy utilities that should only be imported in Node
scripts and never be imported into bundled code like Next.js

### UI library pre-bundling

With this release, we've dramatically sped up the compile time for
Payload by pre-bundling our entire UI package for use inside of the
Payload admin itself. There are new exports that should be used within
Payload custom components:

1. `@payloadcms/ui/client` - all client components 
2. `@payloadcms/ui/server` - all server components

For all of your custom Payload admin UI components, you should be
importing from one of these two pre-compiled barrel files rather than
importing from the more deeply nested exports directly. That will keep
compile times nice and speedy, and will also make sure that the bundled
JS for your admin UI is kept small.

For example, whereas before, if you imported the Payload `Button`, you
would have imported it like this:

```ts
import { Button } from '@payloadcms/ui/elements/Button'
```

Now, you would import it like this:

```ts
import { Button } from '@payloadcms/ui/client'
```

This is a significant DX / performance optimization that we're pretty
pumped about.

However, if you are importing or re-using Payload UI components
_outside_ of the Payload admin UI, for example in your own frontend
apps, you can import from the individual component exports which will
make sure that the bundled JS is kept to a minimum in your frontend
apps. So in your own frontend, you can continue to import directly to
the components that you want to consume rather than importing from the
pre-compiled barrel files.

Individual component exports will now come with their corresponding CSS
and everything will work perfectly as-expected.

### Specific exports have changed

- `'@payloadcms/ui/templates/Default'` and
`'@payloadcms/ui/templates/Minimal`' are now exported from
`'@payloadcms/next/templates'`
- Old: `import { LogOut } from '@payloadcms/ui/icons/LogOut'` new:
`import { LogOutIcon } from '@payloadcms/ui/icons/LogOut'`

## Background info

In effort to make local dev as fast as possible, we need to import as
few files as possible so that the compiler has less to process. One way
we've achieved this in the Admin Panel was to _remove_ all .scss imports
from all components in the `@payloadcms/ui` module using a build
process. This stripped all `import './index.scss'` statements out of
each component before injecting them into `dist`. Instead, it bundles
all of the CSS into a single `main.css` file, and we import _that_ at
the root of the app.

While this concept is _still_ the right solution to the problem, this
particular approach is not viable when using these components outside
the Admin Panel, where not only does this root stylesheet not exist, but
where it would also bloat your app with unused styles. Instead, we need
to _keep_ these .scss imports in place so they are imported directly
alongside your components, as expected. Then, we need create a _new_
build step that _separately_ compiles the components _without_ their
stylesheets—this way your app can consume either as needed from the new
`client` and `server` barrel files within `@payloadcms/ui`, i.e. from
within `@payloadcms/next` and all other admin-specific packages and
plugins.

This way, all other applications will simply import using the direct
file paths, just as they did before. Except now they come with
stylesheets.

And we've gotten a pretty awesome initial compilation performance boost.

---------

Co-authored-by: James <james@trbl.design>
Co-authored-by: Alessio Gravili <alessio@gravili.de>
2024-06-17 14:25:36 -04:00
Dan Ribbens
ff82bb5661 fix(db-postgres)!: create predefined migration missing json schema output (#6641)
## Description

fixes #6630

# BREAKING CHANGES

This only applies to you if you using db-postgres and have created the
`v2-v3-relationships` migration released in
[v3.0.0-beta.39](https://github.com/payloadcms/payload/releases/tag/v3.0.0-beta.39)
from @payloadcms/db-postgres <= v3.0.0-beta.40.

### Steps to fix

- Delete the existing v2-v3-relationships migration file. 
- If changes were made to your config since the previous migration was
made, you will need to revert those by checking out a previous commit in
your version control.
- Recreate the migration using `payload migrate:create --file
@payloadcms/db-postgres/relationships-v2-v3` to make the migration with
the snapshot .json file.
2024-06-05 16:15:01 -04:00
Alessio Gravili
c3589ded08 fix(db-postgres): type issue in migratePostgresV2toV3 call if ts strict mode is enabled (#6585) 2024-05-31 22:37:08 -04:00
Dan Ribbens
edfa85bcd5 feat(db-postgres)!: relationship column (#6339)
BREAKING CHANGE:

Moves `upload` field and `relationship` fields with `hasMany: false` &
`relationTo: string` from the many-to-many `_rels` join table to simple
columns. This only affects Postgres database users.

## TL;DR

We have dramatically simplified the storage of simple relationships in
relational databases to boost performance and align with more expected
relational paradigms. If you are using the beta Postgres adapter, and
you need to keep simple relationship data, you'll need to run a
migration script that we provide you.

### Background

For example, prior to this update, a collection of "posts" with a simple
`hasMany: false` and `relationTo: 'categories'` field would have a
`posts_rels` table where the category relations would be stored.

This was somewhat unnecessary as simple relations like this can be
expressed with a `category_id` column which is configured as a foreign
key. This also introduced added complexity for dealing directly with the
database if all you have are simple relations.

### Who needs to migrate

You need to migrate if you are using the beta Postgres database adapter
and any of the following applies to you.

- If you have versions enabled on any collection / global
- If you use the `upload` field 
- If you have relationship fields that are `hasMany: false` (default)
and `relationTo` to a single category ([has
one](https://payloadcms.com/docs/fields/relationship#has-one)) relations

### We have a migration for you

Even though the Postgres adapter is in beta, we've prepared a predefined
migration that will work out of the box for you to migrate from an
earlier version of the adapter to the most recent version easily.

It makes the schema changes in step with actually moving the data from
the old locations to the new before adding any null constraints and
dropping the old columns and tables.

### How to migrate

The steps to preserve your data while making this update are as follows.
These steps are the same whether you are moving from Payload v2 to v3 or
a previous version of v3 beta to the most recent v3 beta.

**Important: during these steps, don't start the dev server unless you
have `push: false` set on your Postgres adapter.**

#### Step 1 - backup

Always back up your database before performing big changes, especially
in production cases.

#### Step 2 - create a pre-update migration 
Before updating to new Payload and Postgres adapter versions, run
`payload migrate:create` without any other config changes to have a
prior snapshot of the schema from the previous adapter version

#### Step 3 - if you're migrating a dev DB, delete the dev `push` row
from your `payload_migrations` table

If you're migrating a dev database where you have the default setting to
push database changes directly to your DB, and you need to preserve data
in your development database, then you need to delete a `dev` migration
record from your database.

Connect directly to your database in any tool you'd like and delete the
dev push record from the `payload_migrations` table using the following
SQL statement:

```sql
DELETE FROM payload_migrations where batch = -1`
```

#### Step 4 - update Payload and Postgres versions to most recent

Update packages, making sure you have matching versions across all
`@payloadcms/*` and `payload` packages (including
`@payloadcms/db-postgres`)

#### Step 5 - create the predefined migration

Run the following command to create the predefined migration we've
provided:

```
payload migrate:create --file @payloadcms/db-postgres/relationships-v2-v3
```

#### Step 6 - migrate!

Run migrations with the following command: 

```
payload migrate
```

Assuming the migration worked, you can proceed to commit this change and
distribute it to be run on all other environments.

Note that if two servers connect to the same database, only one should
be running migrations to avoid transaction conflicts.

Related discussion:
https://github.com/payloadcms/payload/discussions/4163

---------

Co-authored-by: James <james@trbl.design>
Co-authored-by: PatrikKozak <patrik@payloadcms.com>
2024-05-30 14:09:11 -04:00
James Mikrut
c2571cfdb6 fix(db-postgres): uuid custom db name (#6408)
## Description

Fixes an issue with creating versions when using custom DB names,
`uuid`, and drafts.

---------

Co-authored-by: PatrikKozak <patrik@payloadcms.com>
2024-05-17 14:11:14 -04:00
Dan Ribbens
12c812d379 fix(db-postgres): query with like on id columns (#6414)
When typing into the search input on the list view of a collection, the
`like` operator is used for id which causes an error for postgres. To
fix this we are sanitizing the `like` for number or uuid fields to
instead be an `equals` operator. An alternate solution would have been
to cast the ids to text `id::text` but this would have performence
implications on larger data sets.

---------

Co-authored-by: James <james@trbl.design>
2024-05-17 14:09:44 -04:00
Patrik
e8d1d369cf fix(db-postgres): filter with ID not_in AND queries (#6359)
## Description

v2 PR [here](https://github.com/payloadcms/payload/pull/6358)

- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## Checklist:

- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
2024-05-15 15:10:51 -04:00
James Mikrut
47cd5f4d01 fix(db-postgres): too many clients already, cannot read properties 'transaction' of undefined (#6338) 2024-05-13 12:13:30 -04:00
Dan Ribbens
bd046e2437 fix(db-postgres): use locales suffix (#6032) 2024-04-25 11:07:52 -04:00
Jarrod Flesch
22c53392a3 chore: improves types for payloadRequest (#6012) 2024-04-25 10:23:03 -04:00
James Mikrut
98722dc0fd fix(db-postgres): postgres version id bug (#6026) 2024-04-25 09:23:13 -04:00
James Mikrut
629d7c3263 fix(db-postgres): fully functional dbNames (#6023) 2024-04-24 22:42:24 -04:00
Elliot DeNolf
5731241a5c fix(db-postgres): postgres uuid (#6003)
Co-authored-by: James <james@trbl.design>
2024-04-24 11:59:39 -04:00
James
bc7445ed99 Merge branch 'beta' of github.com:payloadcms/payload into fix/postgres-relation-names 2024-04-23 12:43:32 -04:00
James
e4d024cd0d chore: properly destroys db in postgres 2024-04-23 12:43:25 -04:00
James
1005de8295 fix(db-postgres): shortens relation names 2024-04-23 12:14:01 -04:00
James
ba423ab424 fix: row table names were not being built properly 2024-04-22 15:10:59 -04:00
Dan Ribbens
8945b7a4fa fix(db-postgres): nested groups in nested blocks validation (#5941)
Co-authored-by: Ricardo Domingues <rfdomingues98@gmail.com>
2024-04-21 00:38:55 -04:00
Dan Ribbens
d5ef93b2ba fix(db-postgres): v3 #5938 extra version suffix table names (#5940) 2024-04-20 23:23:06 -04:00
Ritsu
d5cbbc472d feat: add count operation to collections (#5930) 2024-04-20 14:45:44 -04:00
Dan Ribbens
234837ee1d fix: postgres query hasMany in (#5884) 2024-04-16 17:09:43 -04:00
Ritsu
f1b2f767bb fix(db-postgres): validateExistingBlockIsIdentical localized (#5840) 2024-04-15 16:50:28 -04:00
Elliot DeNolf
5f57782199 fix(db-postgres): properly pass id type for type gen (#5859) 2024-04-15 13:38:46 -04:00
Gabriel Novotny
73c76cab77 fix: postgres turbopack static analysis error (#5832) 2024-04-12 18:53:19 -04:00
Dan Ribbens
0d40d87b31 fix(db-postgres): relationship query pagination (#5803) 2024-04-12 11:18:40 -04:00
James
96dbab8834 chore: misc fixes 2024-04-10 16:58:08 -04:00
James
4f9fdb6c14 fix: number ids were not sanitized to number in rest api 2024-04-10 16:01:28 -04:00
Dan Ribbens
f39f95af6d feat: custom db naming for postgres and mongodb (#5697) 2024-04-05 17:09:08 -04:00
Paul
e64660f2d8 fix(db-postgres): querying by localised relations postgres (#5686) 2024-04-05 10:37:58 -04:00
James
9bc56bcfc7 chore: exports sql from pg adapter 2024-04-04 20:30:38 -04:00
Elliot DeNolf
4a11bf956d fix: db migrations esm part 2 (#5677) 2024-04-04 19:36:59 -04:00
Patrik
bf843fe598 fix: skip parsing if operator is exists (#5639) 2024-04-04 14:11:22 -04:00
Dan Ribbens
a1e8c4eb2b fix(db-postgres): query with contains operator hasMany (#5481) 2024-03-26 15:26:46 -04:00