### What?
The `in` & `not_in` operators were not properly working for `text`
fields as this operator requires an array of values for it's input.
### How?
Conditionally renders a multi select input for `text` fields when
filtering by `in` & `not_in` operators.
Trying to understand why bug #12002 arose, I found that both the `sort`
and `hooks` test suites are not running in CI.
I'm adding those 2 suites to the array, though later we should find a
way to automate this so it doesn't happen again. Manually rewriting all
test suites in the GitHub action is error-prone. It's very easy to
forget to add it when creating a new test suite
### What?
In the List View, row data related to images and relationships gets
stuck when you go from one page to another.
### Why?
The `key` we are providing is not unique and not triggering the DOM to
update.
### How?
Uses the `row id` as a unique key prop to each table row to ensure
proper re-rendering of rows during pagination.
#### Testing
Adds e2e test to `upload` test suite. You can recreate the issue using
the `upload` test suite and new `list view preview` collection.
On devices without a top-notch CPU, typing in the rich text editor is
laggy even in the very basic community test suite's "Post" collection.
Lags can be up to multiple seconds. This lag can be reproduced by e.g.
throttling the CPU by 6x on a MacBook Pro with M1 Pro chip and 32GB of
RAM. Typing at regular speed already stutters, and the Chromium
performance monitor shows 100% peak CPU utilization. Under the same
circumstances, the Lexical rich text editor on
https://playground.lexical.dev/ does not exhibit the same laggy UI
reactions.
The issue was narrowed down to the editor state serialization that was
so far executed on every change in `Field.tsx` and utilizing more than 1
frame's worth of CPU time.
This PR attempts to address the issue by asking the browser to queue the
work in moments where it doesn't interfere with UI responsiveness, via
`requestIdleCallback`.
To verify this change, simulate a slow CPU by setting `CPU: 6x slowdown`
in the Chromium `Performance` Dev Tool panel, and then type into the
community test suite's example post's rich text field.
I did not collect exhaustive benchmarks, since numbers are system
specific and the impact of the code change is simple to verify.
Demos:
Before, whole words are not appearing while typing, but then appear all
at once, INP is 6s, and CPU at 100% basically the whole interaction
time:
https://github.com/user-attachments/assets/535653d5-c9e6-4189-a0e0-f71d39c43c31
After: Most letters appear without delay, individual letters can be
slightly delayed, but INP is much more reasonable 350ms, and CPU has
enough bandwidth to drop below 100% utilization:
https://github.com/user-attachments/assets/e627bf50-b441-41de-b3a3-7ba5443ff049⬆️ This recording is from an earlier solution attempt with 500ms
debouncing. The current approach with `requestIdleCallback` increases
CPU usage back to a close 100%, but the INP is further reduced to 2xxms
on my machine, and the perceived UI laggyness is comparable to this
recording.
---
This PR only addresses the rich text editor, because that's where the
performance was a severe usability deal-breaker for real world usage.
Presumably other input fields where users trigger a lot of change events
in succession such as text, textarea, number, and JSON fields might also
benefit from similar debouncing.
Currently, the lexical version diff component is completely unstyled, as
the scss was never included in our css bundle. This PR ensures that the
diff component scss is included in our css bundle
### What?
The `in` & `not_in` operators were not properly working for `number`
fields as this operator requires an array of values for it's input.
### How?
Conditionally renders a multi select input for `number` fields when
filtering by `in` & `not_in` operators.
### What?
The list filters in the collection view allows invalid queries. If you
enter a value and then change operator, the value will remain even if it
doesn't pass the new value field validation, and an error is thrown.
### Why?
The value isn't reset or revalidated on operator change. It is reset on
field change.
### How?
Resets the value field when the operator changes.
Fixes#10648
### What?
UI only issue: An array row with `required: false` and `minRows: x` was
displaying an error banner - this should only happen if one or more rows
are present.
The validation is not affected, the document still saved as expected,
but the error should not be inaccurately displayed.
### Why?
The logic for displaying the `minRow` validation error was `rows.length
> minRows` and it needs to be `rows.length > 1 && rows.length > minRows`
### How?
Updates the UI logic.
Fixes#12010
Previously, querying by polymorphic joins `relationTo` with
`overrideAccess: false` caused an error:
```
QueryError: The following paths cannot be queried: relationTo
```
As this field actually doesn't exist in the schema. Now, under condition
that the query comes from a polymorphic join we skip checking
`relationTo` field access.
Fixes https://github.com/payloadcms/payload/issues/11975
Previously, this configuration was causing errors in postgres due to
long names, even though `dbName` is used:
```
{
slug: 'aliases',
fields: [
{
name: 'thisIsALongFieldNameThatWillCauseAPostgresErrorEvenThoughWeSetAShorterDBName',
dbName: 'shortname',
type: 'array',
fields: [
{
name: 'nested_field_1',
type: 'array',
dbName: 'short_nested_1',
fields: [],
},
{
name: 'nested_field_2',
type: 'text',
},
],
},
],
},
```
This is because we were generating Drizzle relation name (for arrays)
always based on the field path and internally, drizzle uses this name
for aliasing. Now, if `dbName` is present, we use `_{dbName}` instead
for the relation name.
### What?
This PR addresses a bug where image edits (crop, focal point, etc.) were
not persisting correctly in bulk uploads due to shared state logic with
single uploads.
### How?
- The `Upload` component now receives `uploadEdits`, `resetUploadEdits`,
and `updateUploadEdits` as props.
- `Upload_v4` was introduced to encapsulate the actual upload logic,
making it easier to reuse and test.
- The `AddingFilesView` and `EditForm` components are responsible for
injecting the correct `uploadEdits` state, depending on context.
- Avoided unnecessary `useFormsManager` usage in `Upload`.
Fixes#11868
### What?
This PR relaxes the mimeType checks in the thumbnail and file cell
components to accommodate an `adminThumbnail` even if the file is a
non-image. This is useful when, for example, using an `adminThumbnail`
function to retrieve or generate thumbnails for files that are
non-images such as videos.
### Why?
To prioritize an admin thumbnail if/when available on file cells and
upload field thumbnails in both edit and list views.
### How?
By relaxing the mimeType checks in the `Thumbnail` component and instead
lifting that responsibility on the caller of this component. Some of
these checks were not needed as the best-fit helper utility function
will automatically select the thumbnailURL if available or revert to the
original url if no best-fit is found.
Demo of admin thumbnail being loaded on non-image while still selecting
best-fit size for images:
When postgres is used and orderable is enabled, payload cannot update
the docs to set the order correctly. This is because the sort on
postgres pushes `null` values to the top causing unique constraints to
error when two documents are updated to the same _order value.
Fixes https://github.com/payloadcms/payload/issues/11888
Previously, if you had `disableLocalStategy: true` and a custom
`password` field, Payload would still control it in `update.ts` by
deleting. Now, we don't do that in this case, unless we have
`disableLocalStetegy.enableFields: true`.
Significantly optimizes the component rendering strategy within the form
state endpoint by precisely rendering only the fields that require it.
This cuts down on server processing and network response sizes when
invoking form state requests **that manipulate array and block rows
which contain server components**, such as rich text fields, custom row
labels, etc. (results listed below).
Here's a breakdown of the issue:
Previously, when manipulating array and block fields, _all_ rows would
render any server components that might exist within them, including
rich text fields. This means that subsequent changes to these fields
would potentially _re-render_ those same components even if they don't
require it.
For example, if you have an array field with a rich text field within
it, adding the first row would cause the rich text field to render,
which is expected. However, when you add a second row, the rich text
field within the first row would render again unnecessarily along with
the new row.
This is especially noticeable for fields with many rows, where every
single row processes its server components and returns RSC data. And
this does not only affect nested rich text fields, but any custom
component defined on the field level, as these are handled in the same
way.
The reason this was necessary in the first place was to ensure that the
server components receive the proper data when they are rendered, such
as the row index and the row's data. Changing one of these rows could
cause the server component to receive the wrong data if it was not
freshly rendered.
While this is still a requirement that rows receive up-to-date props, it
is no longer necessary to render everything.
Here's a breakdown of the actual fix:
This change ensures that only the fields that are actually being
manipulated will be rendered, rather than all rows. The existing rows
will remain in memory on the client, while the newly rendered components
will return from the server. For example, if you add a new row to an
array field, only the new row will render its server components.
To do this, we send the path of the field that is being manipulated to
the server. The server can then use this path to determine for itself
which fields have already been rendered and which ones need required
rendering.
## Results
The following results were gathered by booting up the `form-state` test
suite and seeding 100 array rows, each containing a rich text field. To
invoke a form state request, we navigate to a document within the
"posts" collection, then add a new array row to the list. The result is
then saved to the file system for comparison.
| Test Suite | Collection | Number of Rows | Before | After | Percentage
Change |
|------|------|---------|--------|--------|--------|
| `form-state` | `posts` | 101 | 1.9MB / 266ms | 80KB / 70ms | ~96%
smaller / ~75% faster |
---------
Co-authored-by: James <james@trbl.design>
Co-authored-by: Alessio Gravili <alessio@gravili.de>
When manipulating array and blocks rows on slow networks, rows can
sometimes disappear and then reappear as requests in the queue arrive.
Consider this scenario:
1. You add a row to form state: this pushes the row in local state
optimistically then triggers a long-running form state request
containing a single row
2. You add another row to form state: this pushes a second row into
local state optimistically then triggers another long-running form state
request containing two rows
3. The first form state request returns with a single row in the
response and replaces local state (which contained two rows)
4. AT THIS MOMENT IN TIME, THE SECOND ROW DISAPPEARS
5. The second form state request returns with two rows in the response
and replaces local state
6. THE UI IS NO LONGER STALE AND BOTH ROWS APPEAR AS EXPECTED
The same issue applies when deleting, moving, and duplicating rows.
Local state becomes out of sync with the form state response and is
ultimately overridden.
The issue is that when we merge the result from form state, we do not
traverse the rows themselves, and instead take the rows in their
entirety. This means that we lose local row state. Instead, we need to
compare the results with what is saved to local state and intelligently
merge them.
### What?
Makes several fields and list item types in query results (e.g. `docs`)
non-nullable.
### Why?
When dealing with code generated from a Payload GraphQL schema, it is
often necessary to use type guards and optional chaining.
For example:
```graphql
type Posts {
docs: [Post]
...
}
```
This implies that the `docs` field itself is nullable and that the array
can contain nulls. In reality, neither of these is true. But because of
the types generated by tools like `graphql-code-generator`, the way to
access `posts` ends up something like this:
```ts
const posts = (query.data.docs ?? []).filter(doc => doc != null);
```
Instead, we would like the schema to be:
```graphql
type Posts {
docs: [Post!]!
...
}
```
### How?
The proposed change involves adding `GraphQLNonNull` where appropriate.
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
<!--
Thank you for the PR! Please go through the checklist below and make
sure you've completed all the steps.
Please review the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository if you haven't already.
The following items will ensure that your PR is handled as smoothly as
possible:
- PR Title must follow conventional commits format. For example, `feat:
my new feature`, `fix(plugin-seo): my fix`.
- Minimal description explained as if explained to someone not
immediately familiar with the code.
- Provide before/after screenshots or code diffs if applicable.
- Link any related issues/discussions from GitHub or Discord.
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Fixes #
-->
### What?
This PR introduces a new `DragOverlay` to the existing `OrderableTable`
component along with a few new utility components. This enables a more
fluid and seamless drag-and-drop experience for end-users who have
enabled `orderable: true` on their collections.
### Why?
Previously, the rows in the `OrderableTable` component were confined
within the table element that renders them. This is troublesome for a
few reasons:
- It clips rows when dragging even slightly outside of the bounds of the
table.
- It creates unnecessary scrollbars within the containing element as the
container is not geared for comprehensive drag-and-drop interactions.
### How?
Introducing a `DragOverlay` component gives the draggable rows an area
to render freely without clipping. This PR also introduces a new
`OrderableRow` (for rendering orderable rows in the table as well as in
a drag preview), and an `OrderableRowDragPreview` component to render a
drag-preview of the active row 1:1 as you would see in the table without
violating HTML rules.
This PR also adds an `onDragStart` event handler to the
`DraggableDroppable` component to allow for listening for the start of a
drag event, necessary for interactions with a `DragOverlay` to
communicate which row initiated the event.
Before:
[orderable-before.webm](https://github.com/user-attachments/assets/ccf32bb0-91db-44f3-8c2a-4f81bb762529)
After:
[orderable-after.webm](https://github.com/user-attachments/assets/d320e7e6-fab8-4ea4-9cb1-38b581cbc50e)
After (With overflow on page):
[orderable-overflow-y.webm](https://github.com/user-attachments/assets/418b9018-901d-4217-980c-8d04d58d19c8)
Fixes https://github.com/payloadcms/payload/issues/11882
Previously, down migration that dropped the `payload_migrations` table
was failing because `migrationTableExists` doesn't check the current
transaction, only in which you can get a `false` value result.
Fixes https://github.com/payloadcms/payload/issues/11901
Previously, when `ValidationError` `errors.path` was referring to a
field with `label` defined as a function, the error message was
generated with `[object Object]`. Now, we call that function instead.
Since the `i18n` argument is required for `StaticLabel`, this PR
introduces so you can pass a partial `req` to `ValidationError` from
which we thread `req.i18n` to the label args.
The same as https://github.com/payloadcms/payload/pull/11763 but also
for GraphQL. The previous fix was working only for the Local API and
REST API due to a different method for querying joins in GraphQL.
### What?
Previously if you used the typescriptSchema and `returned: false`, the
field would still be required anyways.
### Why?
We were adding fields to be required on the collection without comparing
the returned schema from typescriptSchema functions.
### How?
This changes the order of logic so that `requiredFieldNames` on the
collection is only after running and checking the field schema.
Continuation of #11867. When rendering custom fields nested within
arrays or blocks, such as the Lexical rich text editor which is treated
as a custom field, these fields will sometimes disappear when form state
requests are invoked sequentially. This is especially reproducible on
slow networks.
This is different from the previous PR in that this issue is caused by
adding _rows_ back-to-back, whereas the previous issue was caused when
adding a single row followed by a change to another field.
Here's a screen recording demonstrating the issue:
https://github.com/user-attachments/assets/5ecfa9ec-b747-49ed-8618-df282e64519d
The problem is that `requiresRender` is never sent in the form state
request for row 2. This is because the [task
queue](https://github.com/payloadcms/payload/pull/11579) processes tasks
within a single `useEffect`. This forces React to batch the results of
these tasks into a single rendering cycle. So if request 1 sets state
that request 2 relies on, request 2 will never use that state since
they'll execute within the same lifecycle.
Here's a play-by-play of the current behavior:
1. The "add row" event is dispatched
a. This sets `requiresRender: true` in form state
1. A form state request is sent with `requiresRender: true`
1. While that request is processing, another "add row" event is
dispatched
a. This sets `requiresRender: true` in form state
b. This adds a form state request into the queue
1. The initial form state request finishes
a. This sets `requiresRender: false` in form state
1. The next form state request that was queued up in 3b is sent with
`requiresRender: false`
a. THIS IS EXPECTED, BUT SHOULD ACTUALLY BE `true`!!
To fix this this, we need to ensure that the `requiresRender` property
is persisted into the second request instead of overridden. To do this,
we can add a new `serverPropsToIgnore` to form state which is read when
the processing results from the server. So if `requiresRender` exists in
`serverPropsToIgnore`, we do not merge it. This works because we
actually mutate form state in between requests. So request 2 can read
the results from request 1 without going through an additional rendering
cycle.
Here's a play-by-play of the fix:
1. The "add row" event is dispatched
a. This sets `requiresRender: true` in form state
b. This adds a task in the queue to mutate form state with
`requiresRender: true`
1. A form state request is sent with `requiresRender: true`
1. While that request is processing, another "add row" event is
dispatched
a. This sets `requiresRender: true` in form state AND
`serverPropsToIgnore: [ "requiresRender" ]`
c. This adds a form state request into the queue
1. The initial form state request finishes
a. This returns `requiresRender: false` from the form state endpoint BUT
IS IGNORED
1. The next form state request that was queued up in 3c is sent with
`requiresRender: true`
Previously, jobs were executed in FIFO order on MongoDB, and LIFO on
Postgres, with no way to configure this behavior.
This PR makes FIFO the default on both MongoDB and Postgres and
introduces the following new options to configure the processing order
globally or on a queue-by-queue basis:
- a `processingOrder` property to the jobs config
- a `processingOrder` argument to `payload.jobs.run()` to override
what's set in the jobs config
It also adds a new `sequential` option to `payload.jobs.run()`, which
can be useful for debugging.
This replaces usage of our `chainMethods` helper to dynamically chain
queries with [drizzle dynamic query
building](https://orm.drizzle.team/docs/dynamic-query-building).
This is more type-safe, more readable and requires less code
This adds support for running multiple job queue tasks in parallel
within the same workflow while preventing conflicts. Previously, this
would have caused the following issues:
- Job log entries get lost - the final job log is incomplete, despite
all tasks having been executed
- Write conflicts in postgres, leading to unique constraint violation
errors
The solution involves handling job log data updates in a way that avoids
overwriting, and ensuring the final update reflects the latest job log
data. Each job log entry now initializes its own ID, so a given job log
entry’s ID remains the same across multiple, parallel task executions.
## Postgres
In Postgres, we need to enable transactions for the
`payload.db.updateJobs` operation; otherwise, two tasks updating the
same job in parallel can conflict. This happens because Postgres handles
array rows by deleting them all, then re-inserting (rather than
upserting). The rows are stored in a separate table, and the following
scenario can occur:
Op 1: deletes all job log rows
Op 2: deletes all job log rows
Op 1: inserts 200 job log rows
Op 2: insert the same 200 job log rows again => `error: “duplicate key
value violates unique constraint "payload_jobs_log_pkey”`
Because transactions were not used, the rows inserted by Op 1
immediately became visible to Op 2, causing the conflict. Enabling
transactions fixes this. In theory, it can still happen if Op 1 commits
before Op 2 starts inserting (due to the read committed isolation
level), but it should occur far less frequently.
Alongside this change, we should consider inserting the rows using an
upsert (update on conflict), which will get rid of this error
completely. That way, if the insertion of Op 1 is visible to Op 2, Op 2
will simply overwrite it, rather than erroring. Individual job entries
are immutable and job entries cannot be deleted, thus this shouldn't
corrupt any data.
## Mongo
In Mongo, the issue is addressed by ensuring that log row deletions
caused due to different log states in concurrent operations are not
merged back to the client job log, and by making sure the final update
includes all job logs.
There is no duplicate key error in Mongo because the array log resides
in the same document and duplicates are simply upserted. We cannot use
transactions in Mongo, as it appears to lock the document in a way that
prevents reliable parallel updates, leading to:
`MongoServerError: WriteConflict error: this operation conflicted with
another operation. Please retry your operation or multi-document
transaction`
You can access the database name from `sanitizedConfig.db.name`. But
currently, it' not possible to access the db name from the unsanitized
config.
Plugins only have access to the unsanitized config. This change allows
db adapters to return the db name early, which will allow plugins to
conditionally initialize db-specific functionality
### What
This PR improves the `getSafeRedirect` utility to improve security
around open redirect handling.
### How
- Normalizes and decodes the redirect path using `decodeURIComponent`
- Catches malformed encodings with a try/catch fallback
- Blocks open redirects
In 3.0, we made the decision to export all types from the main package
export (e.g. `payload/types` => `payload`). This improves type
discoverability by IDEs and simplifies importing types.
This PR does the same for our db adapters, which still have a separate
`/types` subpath export. While those are kept for
backwards-compatibility, we can remove them in 4.0.
