Significantly optimizes the component rendering strategy within the form
state endpoint by precisely rendering only the fields that require it.
This cuts down on server processing and network response sizes when
invoking form state requests **that manipulate array and block rows
which contain server components**, such as rich text fields, custom row
labels, etc. (results listed below).
Here's a breakdown of the issue:
Previously, when manipulating array and block fields, _all_ rows would
render any server components that might exist within them, including
rich text fields. This means that subsequent changes to these fields
would potentially _re-render_ those same components even if they don't
require it.
For example, if you have an array field with a rich text field within
it, adding the first row would cause the rich text field to render,
which is expected. However, when you add a second row, the rich text
field within the first row would render again unnecessarily along with
the new row.
This is especially noticeable for fields with many rows, where every
single row processes its server components and returns RSC data. And
this does not only affect nested rich text fields, but any custom
component defined on the field level, as these are handled in the same
way.
The reason this was necessary in the first place was to ensure that the
server components receive the proper data when they are rendered, such
as the row index and the row's data. Changing one of these rows could
cause the server component to receive the wrong data if it was not
freshly rendered.
While this is still a requirement that rows receive up-to-date props, it
is no longer necessary to render everything.
Here's a breakdown of the actual fix:
This change ensures that only the fields that are actually being
manipulated will be rendered, rather than all rows. The existing rows
will remain in memory on the client, while the newly rendered components
will return from the server. For example, if you add a new row to an
array field, only the new row will render its server components.
To do this, we send the path of the field that is being manipulated to
the server. The server can then use this path to determine for itself
which fields have already been rendered and which ones need required
rendering.
## Results
The following results were gathered by booting up the `form-state` test
suite and seeding 100 array rows, each containing a rich text field. To
invoke a form state request, we navigate to a document within the
"posts" collection, then add a new array row to the list. The result is
then saved to the file system for comparison.
| Test Suite | Collection | Number of Rows | Before | After | Percentage
Change |
|------|------|---------|--------|--------|--------|
| `form-state` | `posts` | 101 | 1.9MB / 266ms | 80KB / 70ms | ~96%
smaller / ~75% faster |
---------
Co-authored-by: James <james@trbl.design>
Co-authored-by: Alessio Gravili <alessio@gravili.de>
Continuation of #11867. When rendering custom fields nested within
arrays or blocks, such as the Lexical rich text editor which is treated
as a custom field, these fields will sometimes disappear when form state
requests are invoked sequentially. This is especially reproducible on
slow networks.
This is different from the previous PR in that this issue is caused by
adding _rows_ back-to-back, whereas the previous issue was caused when
adding a single row followed by a change to another field.
Here's a screen recording demonstrating the issue:
https://github.com/user-attachments/assets/5ecfa9ec-b747-49ed-8618-df282e64519d
The problem is that `requiresRender` is never sent in the form state
request for row 2. This is because the [task
queue](https://github.com/payloadcms/payload/pull/11579) processes tasks
within a single `useEffect`. This forces React to batch the results of
these tasks into a single rendering cycle. So if request 1 sets state
that request 2 relies on, request 2 will never use that state since
they'll execute within the same lifecycle.
Here's a play-by-play of the current behavior:
1. The "add row" event is dispatched
a. This sets `requiresRender: true` in form state
1. A form state request is sent with `requiresRender: true`
1. While that request is processing, another "add row" event is
dispatched
a. This sets `requiresRender: true` in form state
b. This adds a form state request into the queue
1. The initial form state request finishes
a. This sets `requiresRender: false` in form state
1. The next form state request that was queued up in 3b is sent with
`requiresRender: false`
a. THIS IS EXPECTED, BUT SHOULD ACTUALLY BE `true`!!
To fix this this, we need to ensure that the `requiresRender` property
is persisted into the second request instead of overridden. To do this,
we can add a new `serverPropsToIgnore` to form state which is read when
the processing results from the server. So if `requiresRender` exists in
`serverPropsToIgnore`, we do not merge it. This works because we
actually mutate form state in between requests. So request 2 can read
the results from request 1 without going through an additional rendering
cycle.
Here's a play-by-play of the fix:
1. The "add row" event is dispatched
a. This sets `requiresRender: true` in form state
b. This adds a task in the queue to mutate form state with
`requiresRender: true`
1. A form state request is sent with `requiresRender: true`
1. While that request is processing, another "add row" event is
dispatched
a. This sets `requiresRender: true` in form state AND
`serverPropsToIgnore: [ "requiresRender" ]`
c. This adds a form state request into the queue
1. The initial form state request finishes
a. This returns `requiresRender: false` from the form state endpoint BUT
IS IGNORED
1. The next form state request that was queued up in 3c is sent with
`requiresRender: true`
### What?
This PR aims to deflake the indexed fields e2e test in
`test/fields/collections/Indexed/e2e.spec.ts`.
The issue is that this test is setup in a way where sometimes two toasts
will present themselves in the ui. The second toast assertion will fail
with a strict mode violation as the toast locator will resolve to two
elements.
### Why?
To prevent this test from flaking in ci.
### How?
Adding a new `dismissAfterAssertion` flag to the `assertToastErrors`
helper function which dismisses the toasts. This way, the toasts will
not raise the aforementioned error as they will be dismissed from the
ui.
The logic is handled in a separate loop through such that the assertions
occur first. This is done so that dismissing a toast does not surface
errors due to the order of toasts being shown changing.
Query Presets allow you to save and share filters, columns, and sort
orders for your collections. This is useful for reusing common or
complex filtering patterns and column configurations across your team.
Query Presets are defined on the fly by the users of your app, rather
than being hard coded into the Payload Config.
Here's a screen recording demonstrating the general workflow as it
relates to the list view. Query Presets are not exclusive to the admin
panel, however, as they could be useful in a number of other contexts
and environments.
https://github.com/user-attachments/assets/1fe1155e-ae78-4f59-9138-af352762a1d5
Each Query Preset is saved as a new record in the database under the
`payload-query-presets` collection. This will effectively make them
CRUDable and allows for an endless number of preset configurations. As
you make changes to filters, columns, limit, etc. you can choose to save
them as a new record and optionally share them with others.
Normal document-level access control will determine who can read,
update, and delete these records. Payload provides a set of sensible
defaults here, such as "only me", "everyone", and "specific users", but
you can also extend your own set of access rules on top of this, such as
"by role", etc. Access control is customizable at the operation-level,
for example you can set this to "everyone" can read, but "only me" can
update.
To enable the Query Presets within a particular collection, set
`enableQueryPresets` on that collection's config.
Here's an example:
```ts
{
// ...
enableQueryPresets: true
}
```
Once enabled, a new set of controls will appear within the list view of
the admin panel. This is where you can select and manage query presets.
General settings for Query Presets are configured under the root
`queryPresets` property. This is where you can customize the labels,
apply custom access control rules, etc.
Here's an example of how you might augment the access control properties
with your own custom rule to achieve RBAC:
```ts
{
// ...
queryPresets: {
constraints: {
read: [
{
label: 'Specific Roles',
value: 'specificRoles',
fields: [roles],
access: ({ req: { user } }) => ({
'access.update.roles': {
in: [user?.roles],
},
}),
},
],
}
}
}
```
Related: #4193 and #3092
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
The blocks e2e tests were flaky due to how we conditionally render
fields as they enter the viewport. This prevented Playwright from every
reaching the target element when running
`locator.scrollIntoViewIfNeeded()`. This is especially flaky on pages
with many fields because the page size would continually grow as it was
scrolled.
To fix this there are new `scrollEntirePage` and `waitForPageStability`
helpers. Together, these will ensure that all fields are rendered and
fully loaded before we start testing. An early attempt at this was made
via `page.mouse.wheel(0, 1750)`, but this is an arbitrary pixel value
that is error prone and is not future proof.
These tests were also flaky by an attempt to trigger a form state action
before it was ready to receive events. The fix here is to disable any
buttons while the form is initializing and let Playwright wait for an
interactive state.
Implements a form state task queue. This will prevent onChange handlers
within the form component from processing unnecessarily often, sometimes
long after the user has stopped making changes. This leads to a
potentially huge number of network requests if those changes were made
slower than the debounce rate. This is especially noticeable on slow
networks.
Does so through a new `useQueue` hook. This hook maintains a stack of
events that need processing but only processes the final event to
arrive. Every time a new event is pushed to the stack, the currently
running process is aborted (if any), and that event becomes the next in
the queue. This results in a shocking reduction in the time it takes
between final change to form state and the final network response, from
~1.5 minutes to ~3 seconds (depending on the scenario, see below).
This likely fixes a number of existing open issues. I will link those
issues here once they are identified and verifiably fixed.
Before:
I'm typing slowly here to ensure my changes aren't debounce by the form.
There are a total of 60 characters typed, triggering 58 network requests
and taking around 1.5 minutes to complete after the final change was
made.
https://github.com/user-attachments/assets/49ba0790-a8f8-4390-8421-87453ff8b650
After:
Here there are a total of 69 characters typed, triggering 11 network
requests and taking only about 3 seconds to complete after the final
change was made.
https://github.com/user-attachments/assets/447f8303-0957-41bd-bb2d-9e1151ed9ec3
Fixes https://github.com/payloadcms/payload/issues/11568
### What? Out of sync errors states
- Collaspibles & Tabs were not reporting accurate child error counts
- Arrays could get into a state where they would not update their error
states
- Slight issue with toasts
### Tabs & Collapsibles
The logic for determining matching field paths was not functioning as
intended. Fields were attempting to match with paths such as `_index-0`
which will not work.
### Arrays
The form state was not updating when the server sent back errorPaths.
This PR adds `errorPaths` to `serverPropsToAccept`.
### Toasts
Some toasts could report errors in the form of `my > > error`. This
ensures they will be `my > error`
### Misc
Removes 2 files that were not in use:
- `getFieldStateFromPaths.ts`
- `getNestedFieldState.ts`
### What?
Supersedes https://github.com/payloadcms/payload/pull/11490.
Refactors imports of `formatAdminURL` to import from `payload/shared`
instead of `@payloadcms/ui/shared`. The ui package now imports and
re-exports the function to prevent this from being a breaking change.
### Why?
This makes it easier for other packages/plugins to consume the
`formatAdminURL` function instead of needing to implement their own or
rely on the ui package for the utility.
Maintains column state in the URL. This makes it possible to share
direct links to the list view in a specific column order or active
column state, similar to the behavior of filters. This also makes it
possible to change both the filters and columns in the same rendering
cycle, a requirement of the "list presets" feature being worked on here:
#11330.
For example:
```
?columns=%5B"title"%2C"content"%2C"-updatedAt"%2C"createdAt"%2C"id"%5D
```
The `-` prefix denotes that the column is inactive.
This strategy performs a single round trip to the server, ultimately
simplifying the table columns provider as it no longer needs to request
a newly rendered table for itself. Without this change, column state
would need to be replaced first, followed by a change to the filters.
This would make an unnecessary number of requests to the server and
briefly render the UI in a stale state.
This all happens behind an optimistic update, where the state of the
columns is immediately reflected in the UI while the request takes place
in the background.
Technically speaking, an additional database query in performed compared
to the old strategy, whereas before we'd send the data through the
request to avoid this. But this is a necessary tradeoff and doesn't have
huge performance implications. One could argue that this is actually a
good thing, as the data might have changed in the background which would
not have been reflected in the result otherwise.
When visiting a collection's list view, the nav item corresponding to
that collection correctly appears in an active state, but is still
rendered as an anchor tag. This makes it possible to reload the current
page by simply clicking the link, which is a problem because this
performs an unnecessary server roundtrip. This is especially apparent
when search params exist in the current URL, as the href on the link
does not.
Unrelated: also cleans up leftover code that was missed in this PR:
#11155
Removes all unnecessary `page.waitForURL` methods within e2e tests.
These are unneeded when following a `page.goto` call because the
subsequent page load is already being awaited.
It is only a requirement when:
- Clicking a link and expecting navigation
- Expecting a redirect after a route change
- Waiting for a change in search params
Similar to the goals of #11026. Adds helper utilities to make
interacting with the blocks field easier within e2e tests. This will
also standardize common functionality across tests and reduce the
overall lines of code for each, making them easier to navigate and
digest.
The following helpers are now available:
- `openBlocksDrawer`: self-explanatory
- `addBlock`: opens the blocks drawer and selects the given block
- `reorderBlocks`: similar to `reorderColumn`, moves blocks using the
drag handle
- `removeAllBlocks`: iterates all rows of a given blocks field and
removes them
Fixes https://github.com/payloadcms/payload/issues/11224
Fixes https://github.com/payloadcms/payload/issues/10492
This PR fixes a few weird behaviours when `validate: true` is set on drafts:
- when autosave is on and you submit an invalid form it would get stuck in an infinite loop
- PreventLeave would not trigger for submitted but invalid forms leading to potential data loss
Changes:
- Adds e2e tests for the above scenarios
- Adds a new `isValid` flag on the `Form` context provider to signal globally if the form is in a valid or invalid state
- Components like Autosave will manage this internally since it manages its own submission flow as well
- Adds PreventLeave to Autosave too for when form is invalid meaning data hasn't been actually saved so we want to prevent the user accidentally losing data by reloading or closing the page
The following tests have been added
The `localized` properly was not stripped out of referenced block fields, if any parent was localized. For normal fields, this is done in sanitizeConfig. As the same referenced block config can be used in both a localized and non-localized config, we are not able to strip it out inside sanitizeConfig by modifying the block config.
Instead, this PR had to bring back tedious logic to handle it everywhere the `field.localized` property is accessed. For backwards-compatibility, we need to keep the existing sanitizeConfig logic. In 4.0, we should remove it to benefit from better test coverage of runtime field.localized handling - for now, this is done for our test suite using the `PAYLOAD_DO_NOT_SANITIZE_LOCALIZED_PROPERTY` flag.
If you have multiple blocks that are used in multiple places, this can quickly blow up the size of your Payload Config. This will incur a performance hit, as more data is
1. sent to the client (=> bloated `ClientConfig` and large initial html) and
2. processed on the server (permissions are calculated every single time you navigate to a page - this iterates through all blocks you have defined, even if they're duplicative)
This can be optimized by defining your block **once** in your Payload Config, and just referencing the block slug whenever it's used, instead of passing the entire block config. To do this, the block can be defined in the `blocks` array of the Payload Config. The slug can then be passed to the `blockReferences` array in the Blocks Field - the `blocks` array has to be empty for compatibility reasons.
```ts
import { buildConfig } from 'payload'
import { lexicalEditor, BlocksFeature } from '@payloadcms/richtext-lexical'
// Payload Config
const config = buildConfig({
// Define the block once
blocks: [
{
slug: 'TextBlock',
fields: [
{
name: 'text',
type: 'text',
},
],
},
],
collections: [
{
slug: 'collection1',
fields: [
{
name: 'content',
type: 'blocks',
// Reference the block by slug
blockReferences: ['TextBlock'],
blocks: [], // Required to be empty, for compatibility reasons
},
],
},
{
slug: 'collection2',
fields: [
{
name: 'editor',
type: 'richText',
editor: lexicalEditor({
BlocksFeature({
// Same reference can be reused anywhere, even in the lexical editor, without incurred performance hit
blocks: ['TextBlock'],
})
})
},
],
},
],
})
```
## v4.0 Plans
In 4.0, we will remove the `blockReferences` property, and allow string block references to be passed directly to the blocks `property`. Essentially, we'd remove the `blocks` property and rename `blockReferences` to `blocks`.
The reason we opted to a new property in this PR is to avoid breaking changes. Allowing strings to be passed to the `blocks` property will prevent plugins that iterate through fields / blocks from compiling.
## PR Changes
- Testing: This PR introduces a plugin that automatically converts blocks to block references. This is done in the fields__blocks test suite, to run our existing test suite using block references.
- Block References support: Most changes are similar. Everywhere we iterate through blocks, we have to now do the following:
1. Check if `field.blockReferences` is provided. If so, only iterate through that.
2. Check if the block is an object (= actual block), or string
3. If it's a string, pull the actual block from the Payload Config or from `payload.blocks`.
The exception is config sanitization and block type generations. This PR optimizes them so that each block is only handled once, instead of every time the block is referenced.
## Benchmarks
60 Block fields, each block field having the same 600 Blocks.
### Before:
**Initial HTML:** 195 kB
**Generated types:** takes 11 minutes, 461,209 lines
https://github.com/user-attachments/assets/11d49a4e-5414-4579-8050-e6346e552f56
### After:
**Initial HTML:** 73.6 kB
**Generated types:** takes 2 seconds, 35,810 lines
https://github.com/user-attachments/assets/3eab1a99-6c29-489d-add5-698df67780a3
### After Permissions Optimization (follow-up PR)
Initial HTML: 73.6 kB
https://github.com/user-attachments/assets/a909202e-45a8-4bf6-9a38-8c85813f1312
## Future Plans
1. This PR does not yet deduplicate block references during permissions calculation. We'll optimize that in a separate PR, as this one is already large enough
2. The same optimization can be done to deduplicate fields. One common use-case would be link field groups that may be referenced in multiple entities, outside of blocks. We might explore adding a new `fieldReferences` property, that allows you to reference those same `config.blocks`.
Fixes#10440. When `filterOptions` are set on a relationship field,
those same filters are not applied to the `Filter` component within the
list view. This is because `filterOptions` is not being thread into the
`RelationshipFilter` component responsible for populating the available
options.
To do this, we first need to be resolve the filter options on the server
as they accept functions. Once resolved, they can be prop-drilled into
the proper component and appended onto the client-side "where" query.
Reliant on #11080.
Fixes#9873. The relationship filter in the "where" builder renders
stale values when switching between fields or adding additional "and"
conditions. This was because the `RelationshipFilter` component was not
responding to changes in the `relationTo` prop and failing to reset
internal state when these events took place.
While it sounds like a simple fix, it was actually quite extensive. The
`RelationshipFilter` component was previously relying on a `useEffect`
that had a callback in its dependencies. This was causing the effect to
run uncontrollably using old references. To avoid this, we use the new
`useEffectEvent` approach which allows the underlying effect to run much
more precisely. Same with the `Condition` component that wraps it. We
now run callbacks directly within event handlers as much as possible,
and rely on `useEffectEvent` _only_ for debounced value changes.
This component was also unnecessarily complex...and still is to some
degree. Previously, it was maintaining two separate refs, one to track
the relationships that have yet to fully load, and another to track the
next pages of each relationship that need to load on the next run. These
have been combined into a single ref that tracks both simultaneously, as
this data is interrelated.
This change also does some much needed housekeeping to the
`WhereBuilder` by improving types, defaulting the operator field, etc.
Related: #11023 and #11032
Unrelated: finds a few more instances where the new `addListFilter`
helper from #11026 could be used. Also removes a few duplicative tests.
Previously, data created by other tests was also leaking into unrelated tests, causing them to fail. The new reset-db-between-tests logic added by this PR fixes this.
Additionally, this increases playwright timeouts for CI, and adds a specific timeout override for opening a drawer, as it was incredibly slow in CI
Adds a new `addListFilter` e2e helper. This will help to standardize
this common functionality across all tests that require filtering list
tables and help reduce the overall lines of code within each test file.
Currently, unless a locale is present in the URL search params, the
locale context is instantiated using the default locale until prefs load
in client-side. This causes the locale selector to briefly render in
with the incorrect (default) locale before being replaced by the proper
locale of the request. For example, if the default locale is `en`, and
the page is requested in `es`, the locale selector will flash with
English before changing to the correct locale, even though the page data
itself is properly loaded in Spanish. This is especially evident within
slow networks.
The fix is to query the user's locale preference server-side and thread
it into the locale provider to initialize state. Because search params
are not available within server layouts, we cannot pass the locale param
in the same way, so we rely on the provider itself to read them from the
`useSearchParams` hook. If present, this takes precedence over the
user's preference if it exists.
Since the root page also queries the user's locale preference to
determine the proper locale across navigation, we use React's cache
function to dedupe these function calls and ensure only a single query
is made to the db for each request.
Fixes#10180. When logged in as an unauthorized user who cannot access
the admin panel, the user is unable to log out through the prompted
`/admin/logout` page. This was because that page was using an incorrect
API endpoint, reading from `admin.user` instead of `user.collection`
when formatting the route. This page was also able to get stuck in an
infinite loading state when attempting to log out without any user at
all. Now, public users can properly log out and then back in with
another user who might have access. The messaging around this was also
misleading. Instead of displaying the "Unauthorized, you must be logged
in to make this request" message, we now display a new "Unauthorized,
this user does not have access to the admin panel" message for added
clarity.
This PR allows to have full type safety on `payload.drizzle` with a
single command
```sh
pnpm payload generate:db-schema
```
Which generates TypeScript code with Drizzle declarations based on the
current database schema.
Example of generated file with the website template:
https://gist.github.com/r1tsuu/b8687f211b51d9a3a7e78ba41e8fbf03
Video that shows the power:
https://github.com/user-attachments/assets/3ced958b-ec1d-49f5-9f51-d859d5fae236
We also now proxy drizzle package the same way we do for Lexical so you
don't have to install it (and you shouldn't because you may have version
mismatch).
Instead, you can import from Drizzle like this:
```ts
import {
pgTable,
index,
foreignKey,
integer,
text,
varchar,
jsonb,
boolean,
numeric,
serial,
timestamp,
uniqueIndex,
pgEnum,
} from '@payloadcms/db-postgres/drizzle/pg-core'
import { sql } from '@payloadcms/db-postgres/drizzle'
import { relations } from '@payloadcms/db-postgres/drizzle/relations'
```
Fixes https://github.com/payloadcms/payload/discussions/4318
In the future we can also support types generation for mongoose / raw
mongodb results.
Continuation of #9846 and partial fix for #9774. When setting
`admin.disableListFilter` retroactively, it remains active within the
list filter controls. Same for when the URL search query contains one of
these fields, except this will actually display the _wrong_ field,
falling back to the _first_ field from the config. The fix is to
properly disable the condition for this field if it's an active filter,
while still preventing it from ever rendering as an option within the
field selector itself.
Partial fix for #9774. When `admin.disableListColumn` is set
retroactively, it continues to appear in column state, but shouldn't.
This was because the table column context was not refreshing after HMR
runs, and would instead hold onto these stale columns until the page
itself refreshes. Similarly, this was also a problem when the user had
saved any of these columns to their list preferences, where those prefs
would take precedence despite these properties being set on the
underlying fields. The fix is to filter these columns from all requests
that send them, and ensure local component state properly refreshes
itself.
Closes#9132. When query params are present in the URL, such as after
searching or filtering in the list view, they are not being retained
after navigating back to that view via `history.back()` (i.e. the back
button). This makes it difficult to quickly navigate in and out of
documents from the list view when an underlying search exists. This was
because the `SearchParamsProvider` is stale when the new view renders,
which then replaces the URL with these stale params. The fix here is to
_not_ use the `SearchParamsProvider` at all, and instead use
`next/navigation` directly. Ultimately, this provider should likely be
marked deprecated and then removed in the next major release for this
very reason.
The version diff view at
`/admin/collections/:collection/:id/versions/:version` was not properly
displaying diffs for iterable fields, such as blocks. There were two
main things wrong here:
1. Fields not properly inheriting parent permissions based on the new
sanitized permissions pattern in #7335
1. The diff components were expecting `permissions` but receiving
`fieldPermissions`. This was not picked up by TS because of our use of
dynamic keys when choosing which component to render for that particular
field. We should change this in the future to use a switch case that
explicitly renders each diff component. This way props are strictly
typed.
Closes#9242 and #9365. Autosave-enabled documents rendered within a
drawer were not being properly handled. This was causing multiple draft
documents to be created upon opening the drawer, as well as an empty
document returned from the server function, etc.
Supports bi-directional import/export between MDX <=> Lexical. JSX will
be mapped to lexical blocks back and forth.
This will allow editing our mdx docs in payload while keeping mdx as the
source of truth
---------
Co-authored-by: Germán Jabloñski <43938777+GermanJablo@users.noreply.github.com>
Deprecates `getPayloadHMR` and simplifies this pattern into a single
`import { getPayload } from 'payload'`.
We will still retain the exported `getPayloadHMR` but it now will throw
a deprecation warning with instructions for how to migrate.
### What?
Upgrades mongoose from 6 to latest `v8.8.1`
Fixes https://github.com/payloadcms/payload/issues/9171
### Why?
Compatibilty with Mongodb Atlas
### How?
- Updates deps
- Changed ObjectId from bson-objectid to use `new Type.ObjectId` from
mongoose for compatibility (only inside of db-mongodb)
- Internal type adjustments
https://github.com/payloadcms/payload/discussions/9088
BREAKING CHANGES:
All projects with existing data having versions enabled, or relationship or upload fields will want to create the predefined migration that converts all strings to ObjectIDs where needed. This can be created using `payload migrate:create --file @payloadcms/mongodb/relationships-v2-v3`.
For projects making use of the exposed Models from mongoose, review the
upgrade guides from [v6 to
v7](https://mongoosejs.com/docs/7.x/docs/migrating_to_7.html) and [v7 to
v8](https://mongoosejs.com/docs/migrating_to_8.html) and make
adjustments as needed.
---------
Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
Live preview e2e tests had no CSS when tested against prod.
For all our other tests, we have a separate test/app directory that
imports CSS. Otherwise, the root-level /app directory is used.
For live-preview, we currently always run against test/live-preview/app,
that has no CSS import.
This PR adds a new test/live-preview/prod/app directory that imports CSS
and is used when we run tests against prod.
In order for this to work, I had to make import map generation smarter
Currently, Payload renders all custom components on initial compile of
the admin panel. This is problematic for two key reasons:
1. Custom components do not receive contextual data, i.e. fields do not
receive their field data, edit views do not receive their document data,
etc.
2. Components are unnecessarily rendered before they are used
This was initially required to support React Server Components within
the Payload Admin Panel for two key reasons:
1. Fields can be dynamically rendered within arrays, blocks, etc.
2. Documents can be recursively rendered within a "drawer" UI, i.e.
relationship fields
3. Payload supports server/client component composition
In order to achieve this, components need to be rendered on the server
and passed as "slots" to the client. Currently, the pattern for this is
to render custom server components in the "client config". Then when a
view or field is needed to be rendered, we first check the client config
for a "pre-rendered" component, otherwise render our client-side
fallback component.
But for the reasons listed above, this pattern doesn't exactly make
custom server components very useful within the Payload Admin Panel,
which is where this PR comes in. Now, instead of pre-rendering all
components on initial compile, we're able to render custom components
_on demand_, only as they are needed.
To achieve this, we've established [this
pattern](https://github.com/payloadcms/payload/pull/8481) of React
Server Functions in the Payload Admin Panel. With Server Functions, we
can iterate the Payload Config and return JSX through React's
`text/x-component` content-type. This means we're able to pass
contextual props to custom components, such as data for fields and
views.
## Breaking Changes
1. Add the following to your root layout file, typically located at
`(app)/(payload)/layout.tsx`:
```diff
/* THIS FILE WAS GENERATED AUTOMATICALLY BY PAYLOAD. */
/* DO NOT MODIFY IT BECAUSE IT COULD BE REWRITTEN AT ANY TIME. */
+ import type { ServerFunctionClient } from 'payload'
import config from '@payload-config'
import { RootLayout } from '@payloadcms/next/layouts'
import { handleServerFunctions } from '@payloadcms/next/utilities'
import React from 'react'
import { importMap } from './admin/importMap.js'
import './custom.scss'
type Args = {
children: React.ReactNode
}
+ const serverFunctions: ServerFunctionClient = async function (args) {
+ 'use server'
+ return handleServerFunctions({
+ ...args,
+ config,
+ importMap,
+ })
+ }
const Layout = ({ children }: Args) => (
<RootLayout
config={config}
importMap={importMap}
+ serverFunctions={serverFunctions}
>
{children}
</RootLayout>
)
export default Layout
```
2. If you were previously posting to the `/api/form-state` endpoint, it
no longer exists. Instead, you'll need to invoke the `form-state` Server
Function, which can be done through the _new_ `getFormState` utility:
```diff
- import { getFormState } from '@payloadcms/ui'
- const { state } = await getFormState({
- apiRoute: '',
- body: {
- // ...
- },
- serverURL: ''
- })
+ const { getFormState } = useServerFunctions()
+
+ const { state } = await getFormState({
+ // ...
+ })
```
## Breaking Changes
```diff
- useFieldProps()
- useCellProps()
```
More details coming soon.
---------
Co-authored-by: Alessio Gravili <alessio@gravili.de>
Co-authored-by: Jarrod Flesch <jarrodmflesch@gmail.com>
Co-authored-by: James <james@trbl.design>
### What?
Adds `populate` property to Local API and REST API operations that can
be used to specify `select` for a specific collection when it's
populated
```ts
const result = await payload.findByID({
populate: {
// type safe if you have generated types
posts: {
text: true,
},
},
collection: 'pages',
depth: 1,
id: aboutPage.id,
})
result.relatedPost // only has text and id properties
```
```ts
fetch('https://localhost:3000/api/pages?populate[posts][text]=true') // highlight-line
.then((res) => res.json())
.then((data) => console.log(data))
```
It also overrides
[`defaultPopulate`](https://github.com/payloadcms/payload/pull/8934)
Ensures `defaultPopulate` doesn't affect GraphQL.
### How?
Implements the property for all operations that have the `depth`
argument.
### What?
Fixes support for custom endpoints with `method: 'put'`.
Previously, this didn't work:
```ts
export default buildConfigWithDefaults({
collections: [ ],
endpoints: [
{
method: 'put',
handler: () => new Response(),
path: '/put',
},
],
})
```
### Why?
We supported this in 2.0 and docs are saying that we can use `'put'` as
`method`
https://payloadcms.com/docs/beta/rest-api/overview#custom-endpoints
### How?
Implements the `REST_PUT` export for `@payloadcms/next/routes`, updates
all templates. Additionally, adds tests to ensure root/collection level
custom endpoints with all necessary methods execute properly.
Fixes https://github.com/payloadcms/payload/issues/8807
-->
Adds a jobs queue to Payload.
- [x] Docs, w/ examples for Vercel Cron, additional services
- [x] Type the `job` using GeneratedTypes in `JobRunnerArgs`
(@AlessioGr)
- [x] Write the `runJobs` function
- [x] Allow for some type of `payload.runTask`
- [x] Open up a new bin script for running jobs
- [x] Determine strategy for runner endpoint to either await jobs
successfully or return early and stay open until job work completes
(serverless ramifications here)
- [x] Allow for job runner to accept how many jobs to run in one
invocation
- [x] Make a Payload local API method for creating a new job easily
(payload.createJob) or similar which is strongly typed (@AlessioGr)
- [x] Make `payload.runJobs` or similar (@AlessioGr)
- [x] Write tests for retrying up to max retries for a given step
- [x] Write tests for dynamic import of a runner
The shape of the config should permit the definition of steps separate
from the job workflows themselves.
```js
const config = {
// Not sure if we need this property anymore
queues: {
},
// A job is an instance of a workflow, stored in DB
// and triggered by something at some point
jobs: {
// Be able to override the jobs collection
collectionOverrides: () => {},
// Workflows are groups of tasks that handle
// the flow from task to task.
// When defined on the config, they are considered as predefined workflows
// BUT - in the future, we'll allow for UI-based workflow definition as well.
workflows: [
{
slug: 'job-name',
// Temporary name for this
// should be able to pass function
// or path to it for Node to dynamically import
controlFlowInJS: '/my-runner.js',
// Temporary name as well
// should be able to eventually define workflows
// in UI (meaning they need to be serialized in JSON)
// Should not be able to define both control flows
controlFlowInJSON: [
{
task: 'myTask',
next: {
// etc
}
}
],
// Workflows take input
// which are a group of fields
input: [
{
name: 'post',
type: 'relationship',
relationTo: 'posts',
maxDepth: 0,
required: true,
},
{
name: 'message',
type: 'text',
required: true,
},
],
},
],
// Tasks are defined separately as isolated functions
// that can be retried on fail
tasks: [
{
slug: 'myTask',
retries: 2,
// Each task takes input
// Used to auto-type the task func args
input: [
{
name: 'post',
type: 'relationship',
relationTo: 'posts',
maxDepth: 0,
required: true,
},
{
name: 'message',
type: 'text',
required: true,
},
],
// Each task takes output
// Used to auto-type the function signature
output: [
{
name: 'success',
type: 'checkbox',
}
],
onSuccess: () => {},
onFail: () => {},
run: myRunner,
},
]
}
}
```
### `payload.createJob`
This function should allow for the creation of jobs based on either a
workflow (group of tasks) or an individual task.
To create a job using a workflow:
```js
const job = await payload.createJob({
// Accept the `name` of a workflow so we can match to either a
// code-based workflow OR a workflow defined in the DB
// Should auto-type the input
workflowName: 'myWorkflow',
input: {
// typed to the args of the workflow by name
}
})
```
To create a job using a task:
```js
const job = await payload.createJob({
// Accept the `name` of a task
task: 'myTask',
input: {
// typed to the args of the task by name
}
})
```
---------
Co-authored-by: Alessio Gravili <alessio@gravili.de>
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
Adds `select` which is used to specify the field projection for local
and rest API calls. This is available as an optimization to reduce the
payload's of requests and make the database queries more efficient.
Includes:
- [x] generate types for the `select` property
- [x] infer the return type by `select` with 2 modes - include (`field:
true`) and exclude (`field: false`)
- [x] lots of integration tests, including deep fields / localization
etc
- [x] implement the property in db adapters
- [x] implement the property in the local api for most operations
- [x] implement the property in the rest api
- [x] docs
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
Fixes https://github.com/payloadcms/payload/issues/8013
**BREAKING:**
- Upgrades minimum supported @types/react version from
npm:types-react@19.0.0-rc.0 to npm:types-react@19.0.0-rc.1
- Upgrades minimum supported @types/react-dom version from
npm:types-react-dom@19.0.0-rc.0 to npm:types-react-dom@19.0.0-rc.1
- Upgrades minimum supported react and react-dom version from
19.0.0-rc-06d0b89e-20240801 to 19.0.0-rc-5dcb0097-20240918
- Upgrades minimum supported Next.js version from 15.0.0-canary.104 to
15.0.0-canary.160
---------
Co-authored-by: PatrikKozak <patrik@payloadcms.com>
Co-authored-by: Jacob Fletcher <jacobsfletch@gmail.com>
## Description
- Adds a new "join" field type to Payload and is supported by all database adapters
- The UI uses a table view for the new field
- `db-mongodb` changes relationships to be stored as ObjectIDs instead of strings (for now querying works using both types internally to the DB so no data migration should be necessary unless you're querying directly, see breaking changes for details
- Adds a reusable traverseFields utility to Payload to make it easier to work with nested fields, used internally and for plugin maintainers
```ts
export const Categories: CollectionConfig = {
slug: 'categories',
fields: [
{
name: 'relatedPosts',
type: 'join',
collection: 'posts',
on: 'category',
}
]
}
```
BREAKING CHANGES:
All mongodb relationship and upload values will be stored as MongoDB ObjectIDs instead of strings going forward. If you have existing data and you are querying data directly, outside of Payload's APIs, you get different results. For example, a `contains` query will no longer works given a partial ID of a relationship since the ObjectID requires the whole identifier to work.
---------
Co-authored-by: Jacob Fletcher <jacobsfletch@gmail.com>
Co-authored-by: James <james@trbl.design>
## Description
### TL;DR:
It's currently not possible to render our field components from a server
component because their `field` prop is the original field config, not
the _client_ config which our components require. Currently, the `field`
prop passed into custom fields changes type depending on whether it's a
server or client component, leaving server components without any access
to the client field config or mechanism to acquire it.
This PR passes the client config to all server field components through
a new `clientField` prop. This allows the following in a server
component, which is very similar to how client field components
currently work:
Server component:
```tsx
import { TextField } from '@payloadcms/ui'
import type { TextFieldServerComponent } from 'payload'
export const MyCustomServerField: TextFieldServerComponent = ({ clientField }) => {
return <TextField field={clientField} />
}
```
Client component:
```tsx
'use client'
import { TextField } from '@payloadcms/ui'
import type { TextFieldClientComponent } from 'payload'
export const MyCustomClientField: TextFieldClientComponent = ({ field }) => {
return <TextField field={field} />
}
```
### Full Background
If you have a custom field component, and it's a server component, there
is currently no way to pass the field prop into Payload's client-side
field components.
Here's an example of the problem:
```tsx
import { TextField } from '@payloadcms/ui'
import type { TextFieldServerComponent } from 'payload'
import React from 'react'
export const MyServerComponent: TextFieldServerComponent = (props) => {
const { field } = props
return (
<TextField field={field} /> // This is not possible
)
}
```
The config needs to be transformed into a client config, however,
because of the sheer number of hard-to-find arguments that the
`createClientField` requires, we cannot use it in its raw form.
Here is another example of the problem:
```tsx
import { TextField } from '@payloadcms/ui'
import { createClientField } from '@payloadcms/ui/utilities/createClientField'
import type { TextFieldServerComponent } from 'payload'
import React from 'react'
export const MyServerComponent: TextFieldServerComponent = ({ createClientField }) => {
const clientField = createClientField({...}) // Not a good option bc it requires many hard-to-find args
return (
<TextField field={clientField} />
)
}
```
Theoretically, we could preformat a `createFieldConfig` function so it
can simply be called without arguments:
```tsx
import { TextField } from '@payloadcms/ui'
import type { TextFieldServerComponent } from 'payload'
import React from 'react'
export const MyServerComponent: TextFieldServerComponent = ({ createClientField }) => {
return <TextField field={createClientField()} />
}
```
But this means the field config would be evaluated twice unnecessarily,
including label functions, etc.
The right way to fix this is to simply pass the client config to server
components through a new `clientField` prop:
```tsx
import { TextField } from '@payloadcms/ui'
import type { TextFieldServerComponent } from 'payload'
import React from 'react'
export const MyServerComponent: TextFieldServerComponent = ({ clientField }) => {
return <TextField field={clientField} />
}
```
- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## Checklist:
- [x] Existing test suite passes locally with my changes
- [x] I have made corresponding changes to the documentation
## Description
Currently, you cannot create, delete, or duplicate documents within the
document drawer directly. To create a document within a relationship
field, for example, you must first navigate to the parent field and open
the "create new" drawer. Similarly (but worse), to duplicate or delete a
document, you must _navigate to the parent document to perform these
actions_ which is incredibly disruptive to the content editing workflow.
This becomes especially apparent within the relationship field where you
can edit documents inline, but cannot duplicate or delete them. This PR
supports all document-level actions within the document drawer so that
these actions can be performed on-the-fly without navigating away.
Inline duplication flow on a polymorphic "hasOne" relationship:
https://github.com/user-attachments/assets/bb80404a-079d-44a1-b9bc-14eb2ab49a46
Inline deletion flow on a polymorphic "hasOne" relationship:
https://github.com/user-attachments/assets/10f3587f-f70a-4cca-83ee-5dbcad32f063
- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
## Type of change
- [x] New feature (non-breaking change which adds functionality)
## Checklist:
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
## Description
Reduces the number of client-side requests made by the relationship
field component, and fixes the visual "blink" of the field's value on
initial load. Does so through a new `useIgnoredEffect` hook that allows
this component's effects to be precisely triggered based on whether a
only _subset_ of its dependencies have changed, which looks something
like this:
```tsx
// ...
useIgnoredEffect(() => {
// Do something
}, [deps], [ignoredDeps])
```
"Ignored deps" are still treated as normal dependencies of the
underlying `useEffect` hook, but they do not cause the provided function
to execute. This is useful if you have a list of dependencies that
change often, but need to scope your effect's logic to explicit
dependencies within that list. This is a typical pattern in React using
refs, just standardized within a reusable hook.
This significantly reduces the overall number of re-renders and
duplicative API requests within the relationship field because the
`useEffect` hooks that control the fetching of these related documents
were running unnecessarily often. In the future, we really ought to
leverage the `RelationshipProvider` used in the List View so that we can
also reduce the number of duplicative requests across _unrelated fields_
within the same document.
Before:
https://github.com/user-attachments/assets/ece7c85e-20fb-49f6-b393-c5e9d5176192
After:
https://github.com/user-attachments/assets/9f0a871e-f10f-4fd6-a58b-8146ece288c4
- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
## Type of change
- [x] Bug fix (non-breaking change which fixes an issue)
## Checklist:
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
