### What?
This PR adds support for `where` querying by the join field (don't
confuse with `where` querying of related docs via `joins.where`)
Previously, this didn't work:
```
const categories = await payload.find({
collection: 'categories',
where: { 'relatedPosts.title': { equals: 'my-title' } },
})
```
### Why?
This is crucial for bi-directional relationships, can be used for access
control.
### How?
Implements `where` handling for join fields the same as we do for
relationships. In MongoDB it's not as efficient as it can be, the old PR
that improves it and can be updated later is here
https://github.com/payloadcms/payload/pull/8858
Fixes https://github.com/payloadcms/payload/discussions/9683
This fixes an issue where fields with the name `file` was being
serialized as a top-level field in multipart form data even when the
collection was not upload-enabled. This caused the value of `file` (when
used as a regular field like a text, array, etc.) to be stripped from
the `_payload`.
- Updated `createFormData` to only delete `data.file` and serialize it
at the top level if `docConfig.upload` is defined.
- This prevents unintended loss of `file` field values for non-upload
collections.
The `file` field now remains safely nested in `_payload` unless it's
part of an upload-enabled collection.
This PR adds a new `SchedulePublish` config type on our schedulePublish
configuration in versions from being just boolean.
Two new options are supported:
- `timeFormat` which controls the formatting of the time slots, allowing
users to change from a 12-hour clock to a 24-hour clock (default to 12
hour)
- `timeIntervals` which controls the generated time slots (default 5)
Example configuration:
```
versions: {
drafts: {
schedulePublish: {
timeFormat: 'HH:mm',
timeIntervals: 5,
},
},
},
```
Fixes form state race conditions. Modifying state while a request is in
flight or while the response is being processed could result in those
changes being overridden.
This was happening for a few reasons:
1. Our merge logic was incorrect. We were disregarding local changes to
state that may have occurred while form state requests are pending. This
was because we were iterating over local state, then while building up
new state, we were ignoring any fields that did not exist in the server
response, like this:
```ts
for (const [path, newFieldState] of Object.entries(existingState)) {
if (!incomingState[path]) {
continue
}
// ...
}
```
To fix this, we need to use local state as the source of truth. Then
when the server state arrives, we need to iterate over _that_. If a
field matches in local state, merge in any new properties. This will
ensure all changes to the underlying state are preserved, including any
potential addition or deletions.
However, this logic breaks down if the server might have created _new_
fields, like when populating array rows. This means they, too, would be
ignored. To get around this, there is a new `addedByServer` property
that flags new fields to ensure they are kept.
This new merge strategy also saves an additional loop over form state.
1. We were merging form state based on a mutable ref. This meant that
changes made within one action cause concurrent actions to have dirty
reads. The fix for this is to merge in an isolated manner by copying
state. This will remove any object references. It is generally not good
practice to mutate state without setting it, anyways, as this causes
mismatches between what is rendered and what is in memory.
1. We were merging server form state directly within an effect, then
replacing state entirely. This meant that if another action took place
at the exact moment in time _after_ merge but _before_ dispatch, the
results of that other action would be completely overridden. The fix for
this is to perform the merge within the reducer itself. This will ensure
that we are working with a trustworthy snapshot of state at the exact
moment in time that the action was invoked, and that React can properly
queue the event within its lifecycle.
### What?
The `in` & `not_in` operators were not properly working for `text`
fields as this operator requires an array of values for it's input.
### How?
Conditionally renders a multi select input for `text` fields when
filtering by `in` & `not_in` operators.
Trying to understand why bug #12002 arose, I found that both the `sort`
and `hooks` test suites are not running in CI.
I'm adding those 2 suites to the array, though later we should find a
way to automate this so it doesn't happen again. Manually rewriting all
test suites in the GitHub action is error-prone. It's very easy to
forget to add it when creating a new test suite
### What?
In the List View, row data related to images and relationships gets
stuck when you go from one page to another.
### Why?
The `key` we are providing is not unique and not triggering the DOM to
update.
### How?
Uses the `row id` as a unique key prop to each table row to ensure
proper re-rendering of rows during pagination.
#### Testing
Adds e2e test to `upload` test suite. You can recreate the issue using
the `upload` test suite and new `list view preview` collection.
On devices without a top-notch CPU, typing in the rich text editor is
laggy even in the very basic community test suite's "Post" collection.
Lags can be up to multiple seconds. This lag can be reproduced by e.g.
throttling the CPU by 6x on a MacBook Pro with M1 Pro chip and 32GB of
RAM. Typing at regular speed already stutters, and the Chromium
performance monitor shows 100% peak CPU utilization. Under the same
circumstances, the Lexical rich text editor on
https://playground.lexical.dev/ does not exhibit the same laggy UI
reactions.
The issue was narrowed down to the editor state serialization that was
so far executed on every change in `Field.tsx` and utilizing more than 1
frame's worth of CPU time.
This PR attempts to address the issue by asking the browser to queue the
work in moments where it doesn't interfere with UI responsiveness, via
`requestIdleCallback`.
To verify this change, simulate a slow CPU by setting `CPU: 6x slowdown`
in the Chromium `Performance` Dev Tool panel, and then type into the
community test suite's example post's rich text field.
I did not collect exhaustive benchmarks, since numbers are system
specific and the impact of the code change is simple to verify.
Demos:
Before, whole words are not appearing while typing, but then appear all
at once, INP is 6s, and CPU at 100% basically the whole interaction
time:
https://github.com/user-attachments/assets/535653d5-c9e6-4189-a0e0-f71d39c43c31
After: Most letters appear without delay, individual letters can be
slightly delayed, but INP is much more reasonable 350ms, and CPU has
enough bandwidth to drop below 100% utilization:
https://github.com/user-attachments/assets/e627bf50-b441-41de-b3a3-7ba5443ff049⬆️ This recording is from an earlier solution attempt with 500ms
debouncing. The current approach with `requestIdleCallback` increases
CPU usage back to a close 100%, but the INP is further reduced to 2xxms
on my machine, and the perceived UI laggyness is comparable to this
recording.
---
This PR only addresses the rich text editor, because that's where the
performance was a severe usability deal-breaker for real world usage.
Presumably other input fields where users trigger a lot of change events
in succession such as text, textarea, number, and JSON fields might also
benefit from similar debouncing.
Currently, the lexical version diff component is completely unstyled, as
the scss was never included in our css bundle. This PR ensures that the
diff component scss is included in our css bundle
### What?
The `in` & `not_in` operators were not properly working for `number`
fields as this operator requires an array of values for it's input.
### How?
Conditionally renders a multi select input for `number` fields when
filtering by `in` & `not_in` operators.
### What?
The list filters in the collection view allows invalid queries. If you
enter a value and then change operator, the value will remain even if it
doesn't pass the new value field validation, and an error is thrown.
### Why?
The value isn't reset or revalidated on operator change. It is reset on
field change.
### How?
Resets the value field when the operator changes.
Fixes#10648
### What?
UI only issue: An array row with `required: false` and `minRows: x` was
displaying an error banner - this should only happen if one or more rows
are present.
The validation is not affected, the document still saved as expected,
but the error should not be inaccurately displayed.
### Why?
The logic for displaying the `minRow` validation error was `rows.length
> minRows` and it needs to be `rows.length > 1 && rows.length > minRows`
### How?
Updates the UI logic.
Fixes#12010
Previously, querying by polymorphic joins `relationTo` with
`overrideAccess: false` caused an error:
```
QueryError: The following paths cannot be queried: relationTo
```
As this field actually doesn't exist in the schema. Now, under condition
that the query comes from a polymorphic join we skip checking
`relationTo` field access.
Fixes https://github.com/payloadcms/payload/issues/11975
Previously, this configuration was causing errors in postgres due to
long names, even though `dbName` is used:
```
{
slug: 'aliases',
fields: [
{
name: 'thisIsALongFieldNameThatWillCauseAPostgresErrorEvenThoughWeSetAShorterDBName',
dbName: 'shortname',
type: 'array',
fields: [
{
name: 'nested_field_1',
type: 'array',
dbName: 'short_nested_1',
fields: [],
},
{
name: 'nested_field_2',
type: 'text',
},
],
},
],
},
```
This is because we were generating Drizzle relation name (for arrays)
always based on the field path and internally, drizzle uses this name
for aliasing. Now, if `dbName` is present, we use `_{dbName}` instead
for the relation name.
### What?
This PR addresses a bug where image edits (crop, focal point, etc.) were
not persisting correctly in bulk uploads due to shared state logic with
single uploads.
### How?
- The `Upload` component now receives `uploadEdits`, `resetUploadEdits`,
and `updateUploadEdits` as props.
- `Upload_v4` was introduced to encapsulate the actual upload logic,
making it easier to reuse and test.
- The `AddingFilesView` and `EditForm` components are responsible for
injecting the correct `uploadEdits` state, depending on context.
- Avoided unnecessary `useFormsManager` usage in `Upload`.
Fixes#11868
### What?
This PR relaxes the mimeType checks in the thumbnail and file cell
components to accommodate an `adminThumbnail` even if the file is a
non-image. This is useful when, for example, using an `adminThumbnail`
function to retrieve or generate thumbnails for files that are
non-images such as videos.
### Why?
To prioritize an admin thumbnail if/when available on file cells and
upload field thumbnails in both edit and list views.
### How?
By relaxing the mimeType checks in the `Thumbnail` component and instead
lifting that responsibility on the caller of this component. Some of
these checks were not needed as the best-fit helper utility function
will automatically select the thumbnailURL if available or revert to the
original url if no best-fit is found.
Demo of admin thumbnail being loaded on non-image while still selecting
best-fit size for images:
When postgres is used and orderable is enabled, payload cannot update
the docs to set the order correctly. This is because the sort on
postgres pushes `null` values to the top causing unique constraints to
error when two documents are updated to the same _order value.
Fixes https://github.com/payloadcms/payload/issues/11888
Previously, if you had `disableLocalStategy: true` and a custom
`password` field, Payload would still control it in `update.ts` by
deleting. Now, we don't do that in this case, unless we have
`disableLocalStetegy.enableFields: true`.
Significantly optimizes the component rendering strategy within the form
state endpoint by precisely rendering only the fields that require it.
This cuts down on server processing and network response sizes when
invoking form state requests **that manipulate array and block rows
which contain server components**, such as rich text fields, custom row
labels, etc. (results listed below).
Here's a breakdown of the issue:
Previously, when manipulating array and block fields, _all_ rows would
render any server components that might exist within them, including
rich text fields. This means that subsequent changes to these fields
would potentially _re-render_ those same components even if they don't
require it.
For example, if you have an array field with a rich text field within
it, adding the first row would cause the rich text field to render,
which is expected. However, when you add a second row, the rich text
field within the first row would render again unnecessarily along with
the new row.
This is especially noticeable for fields with many rows, where every
single row processes its server components and returns RSC data. And
this does not only affect nested rich text fields, but any custom
component defined on the field level, as these are handled in the same
way.
The reason this was necessary in the first place was to ensure that the
server components receive the proper data when they are rendered, such
as the row index and the row's data. Changing one of these rows could
cause the server component to receive the wrong data if it was not
freshly rendered.
While this is still a requirement that rows receive up-to-date props, it
is no longer necessary to render everything.
Here's a breakdown of the actual fix:
This change ensures that only the fields that are actually being
manipulated will be rendered, rather than all rows. The existing rows
will remain in memory on the client, while the newly rendered components
will return from the server. For example, if you add a new row to an
array field, only the new row will render its server components.
To do this, we send the path of the field that is being manipulated to
the server. The server can then use this path to determine for itself
which fields have already been rendered and which ones need required
rendering.
## Results
The following results were gathered by booting up the `form-state` test
suite and seeding 100 array rows, each containing a rich text field. To
invoke a form state request, we navigate to a document within the
"posts" collection, then add a new array row to the list. The result is
then saved to the file system for comparison.
| Test Suite | Collection | Number of Rows | Before | After | Percentage
Change |
|------|------|---------|--------|--------|--------|
| `form-state` | `posts` | 101 | 1.9MB / 266ms | 80KB / 70ms | ~96%
smaller / ~75% faster |
---------
Co-authored-by: James <james@trbl.design>
Co-authored-by: Alessio Gravili <alessio@gravili.de>
When manipulating array and blocks rows on slow networks, rows can
sometimes disappear and then reappear as requests in the queue arrive.
Consider this scenario:
1. You add a row to form state: this pushes the row in local state
optimistically then triggers a long-running form state request
containing a single row
2. You add another row to form state: this pushes a second row into
local state optimistically then triggers another long-running form state
request containing two rows
3. The first form state request returns with a single row in the
response and replaces local state (which contained two rows)
4. AT THIS MOMENT IN TIME, THE SECOND ROW DISAPPEARS
5. The second form state request returns with two rows in the response
and replaces local state
6. THE UI IS NO LONGER STALE AND BOTH ROWS APPEAR AS EXPECTED
The same issue applies when deleting, moving, and duplicating rows.
Local state becomes out of sync with the form state response and is
ultimately overridden.
The issue is that when we merge the result from form state, we do not
traverse the rows themselves, and instead take the rows in their
entirety. This means that we lose local row state. Instead, we need to
compare the results with what is saved to local state and intelligently
merge them.
### What?
Makes several fields and list item types in query results (e.g. `docs`)
non-nullable.
### Why?
When dealing with code generated from a Payload GraphQL schema, it is
often necessary to use type guards and optional chaining.
For example:
```graphql
type Posts {
docs: [Post]
...
}
```
This implies that the `docs` field itself is nullable and that the array
can contain nulls. In reality, neither of these is true. But because of
the types generated by tools like `graphql-code-generator`, the way to
access `posts` ends up something like this:
```ts
const posts = (query.data.docs ?? []).filter(doc => doc != null);
```
Instead, we would like the schema to be:
```graphql
type Posts {
docs: [Post!]!
...
}
```
### How?
The proposed change involves adding `GraphQLNonNull` where appropriate.
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
<!--
Thank you for the PR! Please go through the checklist below and make
sure you've completed all the steps.
Please review the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository if you haven't already.
The following items will ensure that your PR is handled as smoothly as
possible:
- PR Title must follow conventional commits format. For example, `feat:
my new feature`, `fix(plugin-seo): my fix`.
- Minimal description explained as if explained to someone not
immediately familiar with the code.
- Provide before/after screenshots or code diffs if applicable.
- Link any related issues/discussions from GitHub or Discord.
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Fixes #
-->
### What?
This PR introduces a new `DragOverlay` to the existing `OrderableTable`
component along with a few new utility components. This enables a more
fluid and seamless drag-and-drop experience for end-users who have
enabled `orderable: true` on their collections.
### Why?
Previously, the rows in the `OrderableTable` component were confined
within the table element that renders them. This is troublesome for a
few reasons:
- It clips rows when dragging even slightly outside of the bounds of the
table.
- It creates unnecessary scrollbars within the containing element as the
container is not geared for comprehensive drag-and-drop interactions.
### How?
Introducing a `DragOverlay` component gives the draggable rows an area
to render freely without clipping. This PR also introduces a new
`OrderableRow` (for rendering orderable rows in the table as well as in
a drag preview), and an `OrderableRowDragPreview` component to render a
drag-preview of the active row 1:1 as you would see in the table without
violating HTML rules.
This PR also adds an `onDragStart` event handler to the
`DraggableDroppable` component to allow for listening for the start of a
drag event, necessary for interactions with a `DragOverlay` to
communicate which row initiated the event.
Before:
[orderable-before.webm](https://github.com/user-attachments/assets/ccf32bb0-91db-44f3-8c2a-4f81bb762529)
After:
[orderable-after.webm](https://github.com/user-attachments/assets/d320e7e6-fab8-4ea4-9cb1-38b581cbc50e)
After (With overflow on page):
[orderable-overflow-y.webm](https://github.com/user-attachments/assets/418b9018-901d-4217-980c-8d04d58d19c8)
Fixes https://github.com/payloadcms/payload/issues/11882
Previously, down migration that dropped the `payload_migrations` table
was failing because `migrationTableExists` doesn't check the current
transaction, only in which you can get a `false` value result.
Fixes https://github.com/payloadcms/payload/issues/11901
Previously, when `ValidationError` `errors.path` was referring to a
field with `label` defined as a function, the error message was
generated with `[object Object]`. Now, we call that function instead.
Since the `i18n` argument is required for `StaticLabel`, this PR
introduces so you can pass a partial `req` to `ValidationError` from
which we thread `req.i18n` to the label args.
The same as https://github.com/payloadcms/payload/pull/11763 but also
for GraphQL. The previous fix was working only for the Local API and
REST API due to a different method for querying joins in GraphQL.
### What?
Previously if you used the typescriptSchema and `returned: false`, the
field would still be required anyways.
### Why?
We were adding fields to be required on the collection without comparing
the returned schema from typescriptSchema functions.
### How?
This changes the order of logic so that `requiredFieldNames` on the
collection is only after running and checking the field schema.
Continuation of #11867. When rendering custom fields nested within
arrays or blocks, such as the Lexical rich text editor which is treated
as a custom field, these fields will sometimes disappear when form state
requests are invoked sequentially. This is especially reproducible on
slow networks.
This is different from the previous PR in that this issue is caused by
adding _rows_ back-to-back, whereas the previous issue was caused when
adding a single row followed by a change to another field.
Here's a screen recording demonstrating the issue:
https://github.com/user-attachments/assets/5ecfa9ec-b747-49ed-8618-df282e64519d
The problem is that `requiresRender` is never sent in the form state
request for row 2. This is because the [task
queue](https://github.com/payloadcms/payload/pull/11579) processes tasks
within a single `useEffect`. This forces React to batch the results of
these tasks into a single rendering cycle. So if request 1 sets state
that request 2 relies on, request 2 will never use that state since
they'll execute within the same lifecycle.
Here's a play-by-play of the current behavior:
1. The "add row" event is dispatched
a. This sets `requiresRender: true` in form state
1. A form state request is sent with `requiresRender: true`
1. While that request is processing, another "add row" event is
dispatched
a. This sets `requiresRender: true` in form state
b. This adds a form state request into the queue
1. The initial form state request finishes
a. This sets `requiresRender: false` in form state
1. The next form state request that was queued up in 3b is sent with
`requiresRender: false`
a. THIS IS EXPECTED, BUT SHOULD ACTUALLY BE `true`!!
To fix this this, we need to ensure that the `requiresRender` property
is persisted into the second request instead of overridden. To do this,
we can add a new `serverPropsToIgnore` to form state which is read when
the processing results from the server. So if `requiresRender` exists in
`serverPropsToIgnore`, we do not merge it. This works because we
actually mutate form state in between requests. So request 2 can read
the results from request 1 without going through an additional rendering
cycle.
Here's a play-by-play of the fix:
1. The "add row" event is dispatched
a. This sets `requiresRender: true` in form state
b. This adds a task in the queue to mutate form state with
`requiresRender: true`
1. A form state request is sent with `requiresRender: true`
1. While that request is processing, another "add row" event is
dispatched
a. This sets `requiresRender: true` in form state AND
`serverPropsToIgnore: [ "requiresRender" ]`
c. This adds a form state request into the queue
1. The initial form state request finishes
a. This returns `requiresRender: false` from the form state endpoint BUT
IS IGNORED
1. The next form state request that was queued up in 3c is sent with
`requiresRender: true`
