Commit Graph

110 Commits

Author SHA1 Message Date
James Mikrut
31b32ef941 feat: deprecates getPayloadHMR in favor of simpler getPayload (#9249)
Deprecates `getPayloadHMR` and simplifies this pattern into a single
`import { getPayload } from 'payload'`.

We will still retain the exported `getPayloadHMR` but it now will throw
a deprecation warning with instructions for how to migrate.
2024-11-16 15:30:05 -05:00
Sasha
b878daf27a feat(db-postgres): deep querying on json and rich text fields (#9102)
### What?
Allows to query on JSON / Rich Text fields in Postgres the same way as
in Mongodb with any nesting level.

Example:
Data:
```js
{
  json: {
    array: [
      {
        text: 'some-text', // nested to array + object
        object: {
          text: 'deep-text', // nested to array + 2x object
          array: [10], // number is nested to array + 2x object + array
        },
      },
    ],
  }
}
```
Query:
```ts
payload.find({
  collection: 'json-fields',
  where: {
    and: [
      {
        'json.array.text': {
          equals: 'some-text',
        },
      },
      {
        'json.array.object.text': {
          equals: 'deep-text',
        },
      },
      {
        'json.array.object.array': {
          in: [10, 20],
        },
      },
      {
        'json.array.object.array': {
          exists: true,
        },
      },
      {
        'json.array.object.notexists': {
          exists: false,
        },
      },
    ],
  },
})
```

### How?
Utilizes [the `jsonb_path_exists` postgres
function](https://www.postgresql.org/docs/current/functions-json.html)
2024-11-12 09:26:04 +02:00
Jacob Fletcher
c96fa613bc feat!: on demand rsc (#8364)
Currently, Payload renders all custom components on initial compile of
the admin panel. This is problematic for two key reasons:
1. Custom components do not receive contextual data, i.e. fields do not
receive their field data, edit views do not receive their document data,
etc.
2. Components are unnecessarily rendered before they are used

This was initially required to support React Server Components within
the Payload Admin Panel for two key reasons:
1. Fields can be dynamically rendered within arrays, blocks, etc.
2. Documents can be recursively rendered within a "drawer" UI, i.e.
relationship fields
3. Payload supports server/client component composition 

In order to achieve this, components need to be rendered on the server
and passed as "slots" to the client. Currently, the pattern for this is
to render custom server components in the "client config". Then when a
view or field is needed to be rendered, we first check the client config
for a "pre-rendered" component, otherwise render our client-side
fallback component.

But for the reasons listed above, this pattern doesn't exactly make
custom server components very useful within the Payload Admin Panel,
which is where this PR comes in. Now, instead of pre-rendering all
components on initial compile, we're able to render custom components
_on demand_, only as they are needed.

To achieve this, we've established [this
pattern](https://github.com/payloadcms/payload/pull/8481) of React
Server Functions in the Payload Admin Panel. With Server Functions, we
can iterate the Payload Config and return JSX through React's
`text/x-component` content-type. This means we're able to pass
contextual props to custom components, such as data for fields and
views.

## Breaking Changes

1. Add the following to your root layout file, typically located at
`(app)/(payload)/layout.tsx`:

    ```diff
    /* THIS FILE WAS GENERATED AUTOMATICALLY BY PAYLOAD. */
    /* DO NOT MODIFY IT BECAUSE IT COULD BE REWRITTEN AT ANY TIME. */
    + import type { ServerFunctionClient } from 'payload'

    import config from '@payload-config'
    import { RootLayout } from '@payloadcms/next/layouts'
    import { handleServerFunctions } from '@payloadcms/next/utilities'
    import React from 'react'

    import { importMap } from './admin/importMap.js'
    import './custom.scss'

    type Args = {
      children: React.ReactNode
    }

+ const serverFunctions: ServerFunctionClient = async function (args) {
    +  'use server'
    +  return handleServerFunctions({
    +    ...args,
    +    config,
    +    importMap,
    +  })
    + }

    const Layout = ({ children }: Args) => (
      <RootLayout
        config={config}
        importMap={importMap}
    +  serverFunctions={serverFunctions}
      >
        {children}
      </RootLayout>
    )

    export default Layout
    ```

2. If you were previously posting to the `/api/form-state` endpoint, it
no longer exists. Instead, you'll need to invoke the `form-state` Server
Function, which can be done through the _new_ `getFormState` utility:

    ```diff
    - import { getFormState } from '@payloadcms/ui'
    - const { state } = await getFormState({
    -   apiRoute: '',
    -   body: {
    -     // ...
    -   },
    -   serverURL: ''
    - })

    + const { getFormState } = useServerFunctions()
    +
    + const { state } = await getFormState({
    +   // ...
    + })
    ```

## Breaking Changes

```diff
- useFieldProps()
- useCellProps()
```

More details coming soon.

---------

Co-authored-by: Alessio Gravili <alessio@gravili.de>
Co-authored-by: Jarrod Flesch <jarrodmflesch@gmail.com>
Co-authored-by: James <james@trbl.design>
2024-11-11 13:59:05 -05:00
Sasha
0a15388edb feat(db-postgres): add point field support (#9078)
### What?
Adds full support for the point field to Postgres and Vercel Postgres
adapters through the Postgis extension. Fully the same API as with
MongoDB, including support for `near`, `within` and `intersects`
operators.

Additionally, exposes to adapter args:
*
`tablesFilter`https://orm.drizzle.team/docs/drizzle-kit-push#including-tables-schemas-and-extensions.
* `extensions` list of extensions to create, for example `['vector',
'pg_search']`, `postgis` is created automatically if there's any point
field

### Why?
It's essential to support that field type, especially if the postgres
adapter should be out of beta on 3.0 stable.

### How?
* Bumps `drizzle-orm` to `0.36.1` and `drizzle-kit` to `0.28.0` as we
need this change https://github.com/drizzle-team/drizzle-orm/pull/3141
* Uses its functions to achieve querying functionality, for example the
`near` operator works through `ST_DWithin` or `intersects` through
`ST_Intersects`.
* Removes MongoDB condition from all point field tests, but keeps for
SQLite

Resolves these discussions:
https://github.com/payloadcms/payload/discussions/8996
https://github.com/payloadcms/payload/discussions/8644
2024-11-11 09:31:47 -05:00
Sasha
4c396c720e fix(db-postgres): alias already in use in this query (#8823)
Fixes https://github.com/payloadcms/payload/issues/8517

Reuses existing joins instead of adding new, duplicate ones which
causes:
```
Error: Alias "" is already used in this query.
```
2024-10-22 10:14:38 -04:00
Sasha
872b205acc fix(drizzle): select hasMany nested to array + tab/group (#8737)
Fixes https://github.com/payloadcms/payload/issues/8732
2024-10-16 21:52:48 -04:00
Sasha
1ffb6c3e13 fix(drizzle): indexes / unique with relationships (#8432)
Fixes https://github.com/payloadcms/payload/issues/8413 and
https://github.com/payloadcms/payload/issues/6460

- Builds indexes for relationships by default in the SQL schema
- Fixes `unique: true` handling with Postgres / SQLite for every type of
relationships (non-polymorphic. hasMany, polymorphic, polymorphic
hasMany) _note_: disables unique for nested to arrays / blocks
relationships in the `_rels` table.
- adds tests

2.0 PR tha ports only indexes creation
https://github.com/payloadcms/payload/pull/8446, because `unique: true`
could be a breaking change if someone has incosistent unique data in the
database.
2024-10-11 13:13:54 -04:00
Patrik
8110cb9956 fix(db-mongodb): properly filters out number field values with the exists operator filter (#8416)
V2 PR [here](https://github.com/payloadcms/payload/pull/8415)
2024-09-30 08:59:58 -04:00
Sasha
613d3b090e fix(drizzle): hasMany / poly relationships nested to localized fields / nested blocks to localized fields (#8456)
fixes https://github.com/payloadcms/payload/issues/8455 and
https://github.com/payloadcms/payload/issues/8462

- Builds the `_locale` column for the `_rels` when it's inside of a
localized group / tab
- Properly builds `sanitizedPath` for blocks in the transform-read
function when it's inside of a localized field. This fixes with fields
inside that have its own table (like `_rels`, select `hasMany: true`
etc)

Adds _more_ tests!
2024-09-28 17:33:50 +03:00
Sasha
338c93a229 fix(drizzle): array/relationship/select hasMany in localized field (#8355)
This PR addresses these issues with localized groups / tabs with
Postgres / SQLite:

- Array fields inside of localized groups. Fixes
https://github.com/payloadcms/payload/issues/8322
- Select fields with `hasMany: true` inside of localized groups. Related
to 1, but still needed its own additional logic.
- Relationship (non-polymorphic / non has-many) inside of localized
groups. Previously, even just trying to define them in the config led to
a crash. Fixes https://github.com/payloadcms/payload/issues/8308

Ensures test coverage for localized groups.
2024-09-23 11:34:02 -04:00
Sasha
2596b85027 fix(drizzle): nested arrays with localized items and versions (#8331)
Fixes https://github.com/payloadcms/payload/issues/7695

Previosuly, trying to append a new item to an array that contains
another array with localized items and enabled versions led to a unique
`_locale` and `_parent_id` error
```ts
{
  name: 'nestedArrayLocalized',
  type: 'array',
  fields: [
    {
      type: 'array',
      name: 'array',
      fields: [
        {
          name: 'text',
          type: 'text',
          localized: true,
        },
      ],
    },
  ],
}
```
2024-09-20 11:16:14 -04:00
Paul
ec3730722b feat(drizzle): add support for in and not_in operators on json field (#8148)
Closes https://github.com/payloadcms/payload/issues/7952

Adds support for `in` and `not_in` operator against JSON field filters.

The following queries are now valid in postgres as well, previously it
only worked in mongo

```ts
await payload.find({
  collection: 'posts',
  where: {
    'data.value': {
      in: ['12', '13', '14'],
    },
  },
  context: {
    disable: true,
  },
})


await payload.find({
  collection: 'posts',
  where: {
    'data.value': {
      not_in: ['12', '13', '14'],
    },
  },
  context: {
    disable: true,
  },
})
```
2024-09-11 11:11:13 -06:00
Sasha
0688c2b79d fix(db-postgres): sanitize tab/group path for table name (#8009)
## Description

Fixes https://github.com/payloadcms/payload/issues/7109

Example of table structures that lead to the problem with camelCased
group / tab names.
`group_field_array_localized` - `groupField` -> `array` (has a localized
field inside)
`group_field_array_nested_array` - `groupField` -> `array` ->
`nestedArray`

<!-- Please include a summary of the pull request and any related issues
it fixes. Please also include relevant motivation and context. -->

- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.

## Type of change

<!-- Please delete options that are not relevant. -->

- [x] Bug fix (non-breaking change which fixes an issue)

## Checklist:

- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
2024-09-06 11:43:47 -04:00
Sasha
3a657847f2 fix(db-postgres): query hasMany text/number in array/blocks (#8003)
## Description

Fixes https://github.com/payloadcms/payload/issues/7671

- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)
## Checklist:

- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
2024-09-04 11:53:43 -04:00
Alessio Gravili
ebd43c7763 feat: pre-compile ui and richtext-lexical with react compiler (#7688)
This noticeably improves performance in the admin panel, for example
when there are multiple richtext editors on one page (& likely
performance in other areas too, though I mainly tested rich text).

The babel plugin currently only optimizes files with a 'use client'
directive at the top - thus we have to make sure to add use client
wherever possible, even if it's imported by a parent client component.

There's one single component that broke when it was compiled using the
React compiler (it stopped being reactive and failed one of our admin
e2e tests):
150808f608
opting out of it completely fixed that issue

Fixes https://github.com/payloadcms/payload/issues/7366
2024-08-19 17:31:36 -04:00
Alessio Gravili
90b7b20699 feat!: beta-next (#7620)
This PR makes three major changes to the codebase:

1. [Component Paths](#component-paths)
Instead of importing custom components into your config directly, they
are now defined as file paths and rendered only when needed. That way
the Payload config will be significantly more lightweight, and ensures
that the Payload config is 100% server-only and Node-safe. Related
discussion: https://github.com/payloadcms/payload/discussions/6938

2. [Client Config](#client-config)
Deprecates the component map by merging its logic into the client
config. The main goal of this change is for performance and
simplification. There was no need to deeply iterate over the Payload
config twice, once for the component map, and another for the client
config. Instead, we can do everything in the client config one time.
This has also dramatically simplified the client side prop drilling
through the UI library. Now, all components can share the same client
config which matches the exact shape of their Payload config (with the
exception of non-serializable props and mapped custom components).

3. [Custom client component are no longer
server-rendered](#custom-client-components-are-no-longer-server-rendered)
Previously, custom components would be server-rendered, no matter if
they are server or client components. Now, only server components are
rendered on the server. Client components are automatically detected,
and simply get passed through as `MappedComponent` to be rendered fully
client-side.

## Component Paths

Instead of importing custom components into your config directly, they
are now defined as file paths and rendered only when needed. That way
the Payload config will be significantly more lightweight, and ensures
that the Payload config is 100% server-only and Node-safe. Related
discussion: https://github.com/payloadcms/payload/discussions/6938

In order to reference any custom components in the Payload config, you
now have to specify a string path to the component instead of importing
it.

Old:

```ts
import { MyComponent2} from './MyComponent2.js'

admin: {
  components: {
    Label: MyComponent2
  },
},
```

New:

```ts
admin: {
  components: {
    Label: '/collections/Posts/MyComponent2.js#MyComponent2', // <= has to be a relative path based on a baseDir configured in the Payload config - NOT relative based on the importing file
  },
},
```

### Local API within Next.js routes

Previously, if you used the Payload Local API within Next.js pages, all
the client-side modules are being added to the bundle for that specific
page, even if you only need server-side functionality.

This `/test` route, which uses the Payload local API, was previously 460
kb. It is now down to 91 kb and does not bundle the Payload client-side
admin panel anymore.

All tests done
[here](https://github.com/payloadcms/payload-3.0-demo/tree/feat/path-test)
with beta.67/PR, db-mongodb and default richtext-lexical:

**dev /admin before:**
![CleanShot 2024-07-29 at 22 49
12@2x](https://github.com/user-attachments/assets/4428e766-b368-4bcf-8c18-d0187ab64f3e)

**dev /admin after:**
![CleanShot 2024-07-29 at 22 50
49@2x](https://github.com/user-attachments/assets/f494c848-7247-4b02-a650-a3fab4000de6)

---

**dev /test before:**
![CleanShot 2024-07-29 at 22 56
18@2x](https://github.com/user-attachments/assets/1a7e9500-b859-4761-bf63-abbcdac6f8d6)

**dev /test after:**
![CleanShot 2024-07-29 at 22 47
45@2x](https://github.com/user-attachments/assets/f89aa76d-f2d5-4572-9753-2267f034a45a)

---

**build before:**
![CleanShot 2024-07-29 at 22 57
14@2x](https://github.com/user-attachments/assets/5f8f7281-2a4a-40a5-a788-c30ddcdd51b5)

**build after::**
![CleanShot 2024-07-29 at 22 56
39@2x](https://github.com/user-attachments/assets/ea8772fd-512f-4db0-9a81-4b014715a1b7)

### Usage of the Payload Local API / config outside of Next.js

This will make it a lot easier to use the Payload config / local API in
other, server-side contexts. Previously, you might encounter errors due
to client files (like .scss files) not being allowed to be imported.

## Client Config

Deprecates the component map by merging its logic into the client
config. The main goal of this change is for performance and
simplification. There was no need to deeply iterate over the Payload
config twice, once for the component map, and another for the client
config. Instead, we can do everything in the client config one time.
This has also dramatically simplified the client side prop drilling
through the UI library. Now, all components can share the same client
config which matches the exact shape of their Payload config (with the
exception of non-serializable props and mapped custom components).

This is breaking change. The `useComponentMap` hook no longer exists,
and most component props have changed (for the better):

```ts
const { componentMap } = useComponentMap() // old
const { config } = useConfig() // new
```

The `useConfig` hook has also changed in shape, `config` is now a
property _within_ the context obj:

```ts
const config = useConfig() // old
const { config } = useConfig() // new
```

## Custom Client Components are no longer server rendered

Previously, custom components would be server-rendered, no matter if
they are server or client components. Now, only server components are
rendered on the server. Client components are automatically detected,
and simply get passed through as `MappedComponent` to be rendered fully
client-side.

The benefit of this change:

Custom client components can now receive props. Previously, the only way
for them to receive dynamic props from a parent client component was to
use hooks, e.g. `useFieldProps()`. Now, we do have the option of passing
in props to the custom components directly, if they are client
components. This will be simpler than having to look for the correct
hook.

This makes rendering them on the client a little bit more complex, as
you now have to check if that component is a server component (=>
already has been rendered) or a client component (=> not rendered yet,
has to be rendered here). However, this added complexity has been
alleviated through the easy-to-use `<RenderMappedComponent />` helper.

This helper now also handles rendering arrays of custom components (e.g.
beforeList, beforeLogin ...), which actually makes rendering custom
components easier in some cases.

## Misc improvements

This PR includes misc, breaking changes. For example, we previously
allowed unions between components and config object for the same
property. E.g. for the custom view property, you were allowed to pass in
a custom component or an object with other properties, alongside a
custom component.

Those union types are now gone. You can now either pass an object, or a
component. The previous `{ View: MyViewComponent}` is now `{ View: {
Component: MyViewComponent} }` or `{ View: { Default: { Component:
MyViewComponent} } }`.

This dramatically simplifies the way we read & process those properties,
especially in buildComponentMap. We can now simply check for the
existence of one specific property, which always has to be a component,
instead of running cursed runtime checks on a shared union property
which could contain a component, but could also contain functions or
objects.

![CleanShot 2024-07-29 at 23 07
07@2x](https://github.com/user-attachments/assets/1e75aa4c-7a4c-419f-9070-216bb7b9a5e5)

![CleanShot 2024-07-29 at 23 09
40@2x](https://github.com/user-attachments/assets/b4c96450-6b7e-496c-a4f7-59126bfd0991)

- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.

---------

Co-authored-by: PatrikKozak <patrik@payloadcms.com>
Co-authored-by: Paul <paul@payloadcms.com>
Co-authored-by: Paul Popus <paul@nouance.io>
Co-authored-by: Jacob Fletcher <jacobsfletch@gmail.com>
Co-authored-by: James <james@trbl.design>
2024-08-13 12:54:33 -04:00
Dan Ribbens
1ec78a16f0 fix(db-postgres): localized array inside blocks field (#7457)
fixes #7371, #5240
2024-07-31 16:33:52 -04:00
Dan Ribbens
695ef32d1e feat(db-*): add defaultValues to database schemas (#7368)
## Description

Prior to this change, the `defaultValue` for fields have only been used
in the application layer of Payload. With this change, you get the added
benefit of having the database columns get the default also. This is
especially helpful when adding new columns to postgres with existing
data to avoid needing to write complex migrations. In MongoDB this
change applies the default to the Mongoose model which is useful when
calling payload.db.create() directly.

This only works for statically defined values.

🙏 A big thanks to @r1tsuu for the feature and implementation idea as I
lifted some code from PR https://github.com/payloadcms/payload/pull/6983

- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.

## Type of change

- [x] New feature (non-breaking change which adds functionality)
- [x] This change requires a documentation update

## Checklist:

- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
- [x] I have made corresponding changes to the documentation
2024-07-30 13:41:18 -04:00
Alessio Gravili
83fd4c6622 chore: run lint and prettier on entire codebase 2024-07-11 15:27:01 -04:00
Dan Ribbens
317bc070e4 fix: cannot use empty strings defaultValue in text-like fields (#6847)
Copy of https://github.com/payloadcms/payload/pull/6842 for beta

Allows empty strings ('') as defaultValue for fields of types: 'text'; 'textarea'; 'email'; 'code'. This can be useful when you want to ensure the value is always a string instead of null/undefined.
2024-06-19 10:02:31 -04:00
Jacob Fletcher
9e76c8f4e3 feat!: prebundle payload, ui, richtext-lexical (#6579)
# Breaking Changes

### New file import locations

Exports from the `payload` package have been _significantly_ cleaned up.
Now, just about everything is able to be imported from `payload`
directly, rather than an assortment of subpath exports. This means that
things like `import { buildConfig } from 'payload/config'` are now just
imported via `import { buildConfig } from 'payload'`. The mental model
is significantly simpler for developers, but you might need to update
some of your imports.

Payload now exposes only three exports:

1. `payload` - all types and server-only Payload code
2. `payload/shared` - utilities that can be used in either the browser
or in Node environments
3. `payload/node` - heavy utilities that should only be imported in Node
scripts and never be imported into bundled code like Next.js

### UI library pre-bundling

With this release, we've dramatically sped up the compile time for
Payload by pre-bundling our entire UI package for use inside of the
Payload admin itself. There are new exports that should be used within
Payload custom components:

1. `@payloadcms/ui/client` - all client components 
2. `@payloadcms/ui/server` - all server components

For all of your custom Payload admin UI components, you should be
importing from one of these two pre-compiled barrel files rather than
importing from the more deeply nested exports directly. That will keep
compile times nice and speedy, and will also make sure that the bundled
JS for your admin UI is kept small.

For example, whereas before, if you imported the Payload `Button`, you
would have imported it like this:

```ts
import { Button } from '@payloadcms/ui/elements/Button'
```

Now, you would import it like this:

```ts
import { Button } from '@payloadcms/ui/client'
```

This is a significant DX / performance optimization that we're pretty
pumped about.

However, if you are importing or re-using Payload UI components
_outside_ of the Payload admin UI, for example in your own frontend
apps, you can import from the individual component exports which will
make sure that the bundled JS is kept to a minimum in your frontend
apps. So in your own frontend, you can continue to import directly to
the components that you want to consume rather than importing from the
pre-compiled barrel files.

Individual component exports will now come with their corresponding CSS
and everything will work perfectly as-expected.

### Specific exports have changed

- `'@payloadcms/ui/templates/Default'` and
`'@payloadcms/ui/templates/Minimal`' are now exported from
`'@payloadcms/next/templates'`
- Old: `import { LogOut } from '@payloadcms/ui/icons/LogOut'` new:
`import { LogOutIcon } from '@payloadcms/ui/icons/LogOut'`

## Background info

In effort to make local dev as fast as possible, we need to import as
few files as possible so that the compiler has less to process. One way
we've achieved this in the Admin Panel was to _remove_ all .scss imports
from all components in the `@payloadcms/ui` module using a build
process. This stripped all `import './index.scss'` statements out of
each component before injecting them into `dist`. Instead, it bundles
all of the CSS into a single `main.css` file, and we import _that_ at
the root of the app.

While this concept is _still_ the right solution to the problem, this
particular approach is not viable when using these components outside
the Admin Panel, where not only does this root stylesheet not exist, but
where it would also bloat your app with unused styles. Instead, we need
to _keep_ these .scss imports in place so they are imported directly
alongside your components, as expected. Then, we need create a _new_
build step that _separately_ compiles the components _without_ their
stylesheets—this way your app can consume either as needed from the new
`client` and `server` barrel files within `@payloadcms/ui`, i.e. from
within `@payloadcms/next` and all other admin-specific packages and
plugins.

This way, all other applications will simply import using the direct
file paths, just as they did before. Except now they come with
stylesheets.

And we've gotten a pretty awesome initial compilation performance boost.

---------

Co-authored-by: James <james@trbl.design>
Co-authored-by: Alessio Gravili <alessio@gravili.de>
2024-06-17 14:25:36 -04:00
Dan Ribbens
edfa85bcd5 feat(db-postgres)!: relationship column (#6339)
BREAKING CHANGE:

Moves `upload` field and `relationship` fields with `hasMany: false` &
`relationTo: string` from the many-to-many `_rels` join table to simple
columns. This only affects Postgres database users.

## TL;DR

We have dramatically simplified the storage of simple relationships in
relational databases to boost performance and align with more expected
relational paradigms. If you are using the beta Postgres adapter, and
you need to keep simple relationship data, you'll need to run a
migration script that we provide you.

### Background

For example, prior to this update, a collection of "posts" with a simple
`hasMany: false` and `relationTo: 'categories'` field would have a
`posts_rels` table where the category relations would be stored.

This was somewhat unnecessary as simple relations like this can be
expressed with a `category_id` column which is configured as a foreign
key. This also introduced added complexity for dealing directly with the
database if all you have are simple relations.

### Who needs to migrate

You need to migrate if you are using the beta Postgres database adapter
and any of the following applies to you.

- If you have versions enabled on any collection / global
- If you use the `upload` field 
- If you have relationship fields that are `hasMany: false` (default)
and `relationTo` to a single category ([has
one](https://payloadcms.com/docs/fields/relationship#has-one)) relations

### We have a migration for you

Even though the Postgres adapter is in beta, we've prepared a predefined
migration that will work out of the box for you to migrate from an
earlier version of the adapter to the most recent version easily.

It makes the schema changes in step with actually moving the data from
the old locations to the new before adding any null constraints and
dropping the old columns and tables.

### How to migrate

The steps to preserve your data while making this update are as follows.
These steps are the same whether you are moving from Payload v2 to v3 or
a previous version of v3 beta to the most recent v3 beta.

**Important: during these steps, don't start the dev server unless you
have `push: false` set on your Postgres adapter.**

#### Step 1 - backup

Always back up your database before performing big changes, especially
in production cases.

#### Step 2 - create a pre-update migration 
Before updating to new Payload and Postgres adapter versions, run
`payload migrate:create` without any other config changes to have a
prior snapshot of the schema from the previous adapter version

#### Step 3 - if you're migrating a dev DB, delete the dev `push` row
from your `payload_migrations` table

If you're migrating a dev database where you have the default setting to
push database changes directly to your DB, and you need to preserve data
in your development database, then you need to delete a `dev` migration
record from your database.

Connect directly to your database in any tool you'd like and delete the
dev push record from the `payload_migrations` table using the following
SQL statement:

```sql
DELETE FROM payload_migrations where batch = -1`
```

#### Step 4 - update Payload and Postgres versions to most recent

Update packages, making sure you have matching versions across all
`@payloadcms/*` and `payload` packages (including
`@payloadcms/db-postgres`)

#### Step 5 - create the predefined migration

Run the following command to create the predefined migration we've
provided:

```
payload migrate:create --file @payloadcms/db-postgres/relationships-v2-v3
```

#### Step 6 - migrate!

Run migrations with the following command: 

```
payload migrate
```

Assuming the migration worked, you can proceed to commit this change and
distribute it to be run on all other environments.

Note that if two servers connect to the same database, only one should
be running migrations to avoid transaction conflicts.

Related discussion:
https://github.com/payloadcms/payload/discussions/4163

---------

Co-authored-by: James <james@trbl.design>
Co-authored-by: PatrikKozak <patrik@payloadcms.com>
2024-05-30 14:09:11 -04:00
James
1005de8295 fix(db-postgres): shortens relation names 2024-04-23 12:14:01 -04:00
Kendell Joseph
3af3a91c87 feat: json field schemas (#5898) 2024-04-19 13:35:59 -04:00
Dan Ribbens
234837ee1d fix: postgres query hasMany in (#5884) 2024-04-16 17:09:43 -04:00
Alessio Gravili
1e60250670 chore: enable all fields int tests (#5603) 2024-04-02 17:00:32 -04:00
Alessio Gravili
f6d2dd520c Merge pull request #5561 from payloadcms/temp38
fix: test suite & transactions fixes
2024-04-02 16:00:10 -04:00
Dan Ribbens
825ca94080 fix: duplicate handles locales with unique (#5600)
* fix: duplicate errors with localized and unique fields

* docs: beforeDuplicate hooks
2024-04-02 15:30:49 -04:00
James
bb8a57d2e9 chore: better pattern to initialize memory server 2024-04-01 17:04:05 -04:00
Dan Ribbens
2fb265ae2d chore: fix broken fields test (#5492) 2024-03-27 12:39:09 -04:00
Alessio Gravili
5241c38ba0 chore: speed up tests by not running seed twice for the first test, and reduce flakiness of lexical e2e test suite 2024-03-25 23:55:42 -04:00
Dan Ribbens
ed01ee1e2d feat: duplicate doc moved from frontend to backend concern (#5342)
BREAKING CHANGE: collection.admin.hooks.beforeDuplicate removed and instead should be handled using field beforeDuplicate hooks which take the full field hook arguments.

* feat: duplicate doc moved from frontend to backend concern

* feat: default beforeDuplicate hook functions on unique fields

* docs: beforeDuplicate field hook

* test: duplicate doc local api

* chore: fix build errors

* chore: add access.create call to duplicate operation

* chore: perfectionist reorder imports
2024-03-19 11:25:19 -04:00
Elliot DeNolf
1ac76d7758 chore: more linting 2024-03-19 01:15:25 -04:00
Elliot DeNolf
c5ecf48d94 chore: add test/ to workspace, update most references 2024-03-19 00:59:56 -04:00
Elliot DeNolf
7ddb68b70d test: fix some easy eslint errors 2024-03-16 10:15:09 -04:00
Elliot DeNolf
14eb66c87d test: refactor int tests to use initPayloadInt which reduces boilerplate 2024-03-16 10:11:00 -04:00
Alessio Gravili
6789e61488 chore: run lint & prettier on everything 2024-03-14 23:53:47 -04:00
Dan Ribbens
d193c677c7 chore: attach mongoMemoryServer to db and destroy in tests (#5326)
* chore: attach mongoMemoryServer to db and destroy in tests

* bump mongodb-memory-server to 9.x

---------

Co-authored-by: Paul Popus <paul@nouance.io>
2024-03-14 15:41:20 -04:00
Alessio Gravili
8895f6420f chore: fix all esm test suite imports 2024-03-08 14:42:24 -05:00
Dan Ribbens
b9dec2f714 Chore/next poc merge main (#5204)
* wip moves payload, user and data into partial req

* chore: adjust req type

* chore(next): installs sass and resolves type errors

* feat: working login route/view

* fix: me route

* chore(next): scaffolds access routes (#4562)

* chore(next): scaffolds admin layout and dashboard view (#4566)

* chore(next): builds initPage utility (#4589)

* feat(3.0): next route handlers (#4590)

* chore: removes old files

* chore(next): ssr list view (#4594)

* chore: removes old files

* chore: adjusts graphql file imports to align with new operation exports

* chore: allows for custom endpoints

* chore: cleanup

* chore(next): ssr edit view (#4614)

* chore(ui): ssr main nav (#4619)

* chore(next): ssr account view (#4620)

* chore(next): ssr auth views and document create (#4631)

* chore(next): ssr globals view (#4640)

* chore(next): scaffolds document layout (#4644)

* chore(next): ssr versions view (#4645)

* chore(next): ssr field conditions (#4675)

* chore(next): ssr field validations (#4700)

* chore(next): moves dashboard view into next dir

* chore(next): moves account view into next dir

* chore(next): moves global edit view into next dir

* chore(next): returns isolated configs and locale from initPage

* chore(next): ssr api view (#4721)

* feat: adds i18n functionality within Rest API, Local and Client contexts (#4749)

* chore: separate client translation groups with empty line

* chore: add missing translation used in db adapters

* chore: simplify next/routes export and import paths

* chore: renames PayloadT to Payload

* chore(next): custom views (#4748)

* chore: fix translation tsconfig

* chore: adjust other package ts-configs that rely on translations

* chore(next): installs @payloadcms/ui as direct dependency

* chore(next): progress to build

* chore(next): migrates types (#4792)

* fixes acccept-language detection

* chore(next): moves remaining components out from payload core (#4794)

* chore(deps): removes all unused dependencies from payload core (#4797)

* chore(next): achieves buildable state (#4803)

* adds Translation component and removes more react-i18next

* fixes up remaining translation strings

* fixes a few i18n TODO's

* chore: remaining translation strings without colons

* chore: adds missing ja translations

* chore(next): ssr group field (#4830)

* chore: removes placeholder t function

* chore: removes old file

* chore(bundler-webpack): removes webpack bundler

* chore(bundler-vite): removes vite bundler

* chore(next): ssr tabs field (#4863)

* chore(next): ssr row field

* chore(next): ssr textarea field

* chore(next): wires server action into document edit view (#4873)

* chore(next): conditional logic (#4880)

* chore(next): ssr radio, point, code, json, ui, and hidden fields (#4891)

* chore(next): ssr collapsible field (#4894)

* chore: remove findByID from req

* chore: adjusts file property on request type

* comment clarification

* chore: wires up busboy with Requst readstream

* chore: ports over express-fileupload into a NextJS compatible format

* chore: adjust upload file structure

* chore: adds try/catch around routes, corrects a few route responses

* chore: renames file/function

* chore: improve req type safety in local operations, misc req.files replacements

* chore: misc type and fn export changes

* chore: ensures root routes take pass unmodified request to root routes

* chore: improve types

* chore: consolidates locale api req initialization (#4922)

* chore(next): overhauls field rendering strategy (#4924)

* chore(next): ssr array field (#4937)

* chore(next): ssr blocks field (#4942)

* chore(next): ssr upload field and document drawer (#4957)

* chore(next): wires form submissions (#4982)

* chore: api handler adjustments

* feat: adds graphql playground handler

* adds credentials include setting to playground

* remove old playground init, stub graphql handler location

* fix: allow for null fallbackLocale

* fix: correctly prioritize locales passed as null

* chore: move all graphql code into next package

* graphql changes

* chore: semi working version of graphql http layer

* gql fix attempts

* rm console log

* chore: partial gql changes

* chore: adds gql and gql-http back into payload

* chore: removes collection from req

* chore: separates graphql package out for schema generation

* chore: dep cleanup

* chore: move graphql handlers

* chore: removes unused deps

* chore(next): ssr list view (#5032)

* chore: refactor response handler order for custom endpoints

* chore: add back in condition for collection GET path with 2 slugs

* chore: rm optional chain

* chore: import sort route file

* chore: allows custom endpoints to attempt before erroring

* feat: adds memoization to translation functions (#5036)

* chore: fix APIError import

* chore: return attemptCustomEndpointBeforeError responses

* chore(next): properly instantiates table columns

* fix(next): attaches params to req and properly assigns prefs key (#5042)

* chore: reorganize next route order

* chore(next): adds RouteError handler to next routes

* chore: builds payload successfully

* chore: misc file omissions

* fix(ui): maintains proper column order

* fix(ui): ensures first cell is a link

* fix(next): properly copies url object in createPayloadRequest (#5064)

* fix(ui): bumps react-toastify to v10.0.4 to fix hydration warnings

* feat: add route for static file GET requests (#5065)

* chore(next): allows resolved config promise to be thread through initPage (#5071)

* chore(ui): conditionally renders field label from props

* feat(next): next install script

* chore: pass config to route handlers

* feat: initial test suite framework (#4929)

* chore(next): renderable account, api, and create first user views (#5084)

* fix(next): properly parses search params in find, update, and delete handlers (#5088)

* chore(next): ssr versions view (#5085)

* chore: adds homepage for scss testing

* chore: moves dev folder to top, establishes new test pattern

* chore: working turbopack

* chore: sets up working dynamic payload-config imports

* remove unused code

* chore: rm console log

* misc

* feat: correctly subs out ability to boot REST API within same process

* chore: WIP dev suites

* chore: removes need for REST_API folder in test dir

* removes duplicate bootAdminPanel fn

* misc

* specify default export

* chore: sets up jest to work with next/jest

* chore: progress to mongodb and sharp builds

* chore: passing community tests

* chore: sorta workin

* chore: adjust payload-config import

* chore: adds rest client for Next handlers

* chore: removes test garb

* chore: restores payload-config tsconfig path temporarily

* chore: establishes pattern for memory db during tests

* chore: bumps mongoose to 7

* chore(next): 404s on nested create urls

* chore: functional _community e2e

* chore: increases e2e expect timeout

* fix(next): sanitizes locale toString from client config

* chore: type fixes

* chore: pulls mongodb from main

* chore: uses graphql to log user in

* feat: passing auth test suite

* chore(ui): threads params through context and conditionally renders document tabs (#5094)

* feat(ui): adds params context (#5095)

* chore: removes unecessary memory allocation for urlPropertiesObject object

* chore: passing graphql test suite

* chore: removes references to bson

* chore: re-enables mongodb memory server for auth test suite

* chore: replace bson with bson-objectid

* feat: passing collections-rest int suite

* chore: fixes bad imports

* chore: more passing int suites

* feat: passing globals int tests

* feat: passing hooks int test suite

* chore: remove last express file

* chore: start live-preview int test migration

* chore: passing localization int tests

* passing relationships int tests

* chore: partial passing upload int tests

* chore: fixes scss imports

* chore(ui): renders document info provider at root (#5106)

* chore: adds schema path to useFieldPath provider, more passing tests

* chore: begins work to optimize translation imports

* chore: add translations to ui ts-config references

* chore: add exports folder to package json exports

* chore: adds readme how-to-use instructions

* chore: attempts refactor of translation imports

* chore: adds authentication:account translation key to server keys

* chore: finishes translation optimization

* chore: ignores warnings from mongodb

* chore(ui): renders live document title (#5115)

* chore(ui): ssr document tabs (#5116)

* chore: handles redirecting from login

* chore: handle redirect with no searchParams

* chore: handle missing segments

* chore(next): migrates server action into standalone api endpoint (#5122)

* chore: adjust dashboard colection segments

* test: update e2e suites

* fix(ui): prevents unnecessary calls to form state

* chore: fix finding global config fields from schema path

* fix(next): executes root POST endpoints

* chore(ui): ignores values returned by form state polling

* chore: scaffolds ssr rte

* chore: renders client leaves

* chore: server-side rendered rich text elements

* chore: defines ClientFunction pattern

* chore(ui): migrates relationship field

* chore: adds translations, cleans up slate

* chore: functional slate link

* chore: slate upload ssr

* chore: relationship slate ssr

* chore: remaining slate ssr

* chore: fixes circular workspace dep

* chore: correct broken int test import paths

* chore: remove media files from root

* chore: server renders custom edit view

* fix(ui): resolves infinite loading in versions view

* fix(next): resolves global edit view lookup

* chore: payload builds

* chore: delete unused files

* chore: removes local property from payload

* chore: adds mongodb as dev dep in db-mongodb package

* chore: hide deprecation warnings for tempfile and jest-environment-jsdom

* chore: remove all translations from translations dist

* chore: clean ts-config files

* chore: simple type fixes

* chore(ui): server renders custom list view

* chore: fix next config payload-config alias

* chore: adds turbo alias paths

* chore: adjusts translation generation

* chore: improve auth function

* chore: eslint config for packages/ui

* chore(ui): exports FormState

* chore(next): migrates account view to latest patterns

* chore: disable barbie mode

* chore(ui): lints

* chore(next): lints

* chore: for alexical

* chore: custom handler type signature adjustment

* fix: non-boolean condition result causes infinite looping (#4579)

* chore(richtext-lexical): upgrade lexical from v0.12.5 to v0.12.6 (#4732)

* chore(richtext-lexical): upgrade all lexical packages from 0.12.5 to 0.12.6

* fix(richtext-lexical): fix TypeScript errors

* fix indenting

* feat(richtext-lexical): Blocks: generate type definitions for blocks fields (#4529)

* feat(richtext-lexical)!: Update lexical from 0.12.6 to 0.13.1, port over all useful changes from playground (#5066)

* feat(richtext-lexical): Update lexical from 0.12.6 to 0.13.1, port over all useful changes from playground

* chore: upgrade lexical version used in monorepo

* chore: remove the 3

* chore: upgrade nodemon versions (#5059)

* feat: add more options to addFieldStatePromise so that it can be used for field flattening (#4799)

* feat(plugin-seo)!: remove support for payload <2.7.0 (#4765)

* chore(plugin-seo): remove test script from package.json (#4762)

* chore: upgrade @types/nodemailer from v6.4.8 to v6.4.14 (#4733)

* chore: revert auth and initPage changes

* chore(next): moves edit and list views (#5170)

* fix: "The punycode module is deprecated" warning by updating nodemailer

* chore: adjust translations tsconfig paths in root

* chore: fix merge build

---------

Co-authored-by: Jarrod Flesch <jarrodmflesch@gmail.com>
Co-authored-by: Jacob Fletcher <jacobsfletch@gmail.com>
Co-authored-by: Jarrod Flesch <30633324+JarrodMFlesch@users.noreply.github.com>
Co-authored-by: Elliot DeNolf <denolfe@gmail.com>
Co-authored-by: James <james@trbl.design>
Co-authored-by: Alessio Gravili <alessio@gravili.de>
Co-authored-by: Alessio Gravili <70709113+AlessioGr@users.noreply.github.com>
2024-02-28 13:44:17 -05:00
Jarrod Flesch
a5e2fa80e8 chore: adds schema path to useFieldPath provider, more passing tests 2024-02-20 15:56:11 -05:00
Jarrod Flesch
a1d66b83e0 fix: disallow duplicate fieldNames to be used on the same level in the config (#4381) 2023-12-11 16:52:24 -05:00
Dan Ribbens
6e9ae65374 fix(db-postgres): querying nested blocks fields (#4404)
Co-authored-by: Jessica Chowdhury <jessica@trbl.design>
2023-12-07 09:18:48 -05:00
Jarrod Flesch
cd07873fc5 fix(db-postgres): allow for nested block fields to be queried (#4237)
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2023-11-29 11:03:57 -05:00
Dan Ribbens
3514bfbdae fix(db-postgres): error saving nested arrays with versions (#4302) 2023-11-29 10:10:08 -05:00
Jarrod Flesch
dbc4ce71e6 chore: fixes test suites that use clearAndSeedEverything (#4080) 2023-11-09 12:30:19 -05:00
Alessio Gravili
fff377ad22 fix(richtext-lexical): Blocks: unnecessary saving node value when initially opening a document & new lexical tests (#4059)
* chore: new lexical int tests and working test structure

* chore: more int tests, and better lexical collection structure

* fix(richtext-lexical): Blocks: unnecessary saving node value when initially opening a document
2023-11-08 21:32:43 +01:00
Hulpoi George-Valentin
266c3274d0 feat: Custom Error, Label, and before/after field components (#3747)
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2023-11-08 14:40:31 -05:00
Alessio Gravili
17f7b94555 chore: improve test suites, upgrade jest and playwright, add debug utilities for lexical (#4011)
* feat(richtext-lexical): 'bottom' position value for plugins

* feat: TestRecorderFeature

* chore: restructuring to seed and clear db before each test

* chore: make sure all tests pass

* chore: make sure indexes are created in seed.ts - this fixes one erroring test

* chore: speed up test runs through db snapshots

* chore: support drizzle when resetting db

* chore: simplify seeding process, by moving boilerplate db reset / snapshot logic into a wrapper function

* chore: add new seeding process to admin test suite

* chore(deps): upgrade jest and playwright

* chore: make sure mongoose-specific tests are not skipped

* chore: fix point test, which was depending on another test (that's bad!)

* chore: fix incorrect import

* chore: remove unnecessary comments

* chore: clearly label lexicalE2E test file as todo

* chore: simplify seed logic

* chore: move versions test suite to new seed system
2023-11-06 16:38:40 +01:00
Dan Ribbens
3404bab83f fix: generate new block ids on create (#3871) 2023-10-27 13:32:20 -04:00