The same as https://github.com/payloadcms/payload/pull/11763 but also
for GraphQL. The previous fix was working only for the Local API and
REST API due to a different method for querying joins in GraphQL.
### What?
Previously if you used the typescriptSchema and `returned: false`, the
field would still be required anyways.
### Why?
We were adding fields to be required on the collection without comparing
the returned schema from typescriptSchema functions.
### How?
This changes the order of logic so that `requiredFieldNames` on the
collection is only after running and checking the field schema.
Continuation of #11867. When rendering custom fields nested within
arrays or blocks, such as the Lexical rich text editor which is treated
as a custom field, these fields will sometimes disappear when form state
requests are invoked sequentially. This is especially reproducible on
slow networks.
This is different from the previous PR in that this issue is caused by
adding _rows_ back-to-back, whereas the previous issue was caused when
adding a single row followed by a change to another field.
Here's a screen recording demonstrating the issue:
https://github.com/user-attachments/assets/5ecfa9ec-b747-49ed-8618-df282e64519d
The problem is that `requiresRender` is never sent in the form state
request for row 2. This is because the [task
queue](https://github.com/payloadcms/payload/pull/11579) processes tasks
within a single `useEffect`. This forces React to batch the results of
these tasks into a single rendering cycle. So if request 1 sets state
that request 2 relies on, request 2 will never use that state since
they'll execute within the same lifecycle.
Here's a play-by-play of the current behavior:
1. The "add row" event is dispatched
a. This sets `requiresRender: true` in form state
1. A form state request is sent with `requiresRender: true`
1. While that request is processing, another "add row" event is
dispatched
a. This sets `requiresRender: true` in form state
b. This adds a form state request into the queue
1. The initial form state request finishes
a. This sets `requiresRender: false` in form state
1. The next form state request that was queued up in 3b is sent with
`requiresRender: false`
a. THIS IS EXPECTED, BUT SHOULD ACTUALLY BE `true`!!
To fix this this, we need to ensure that the `requiresRender` property
is persisted into the second request instead of overridden. To do this,
we can add a new `serverPropsToIgnore` to form state which is read when
the processing results from the server. So if `requiresRender` exists in
`serverPropsToIgnore`, we do not merge it. This works because we
actually mutate form state in between requests. So request 2 can read
the results from request 1 without going through an additional rendering
cycle.
Here's a play-by-play of the fix:
1. The "add row" event is dispatched
a. This sets `requiresRender: true` in form state
b. This adds a task in the queue to mutate form state with
`requiresRender: true`
1. A form state request is sent with `requiresRender: true`
1. While that request is processing, another "add row" event is
dispatched
a. This sets `requiresRender: true` in form state AND
`serverPropsToIgnore: [ "requiresRender" ]`
c. This adds a form state request into the queue
1. The initial form state request finishes
a. This returns `requiresRender: false` from the form state endpoint BUT
IS IGNORED
1. The next form state request that was queued up in 3c is sent with
`requiresRender: true`
Previously, jobs were executed in FIFO order on MongoDB, and LIFO on
Postgres, with no way to configure this behavior.
This PR makes FIFO the default on both MongoDB and Postgres and
introduces the following new options to configure the processing order
globally or on a queue-by-queue basis:
- a `processingOrder` property to the jobs config
- a `processingOrder` argument to `payload.jobs.run()` to override
what's set in the jobs config
It also adds a new `sequential` option to `payload.jobs.run()`, which
can be useful for debugging.
This replaces usage of our `chainMethods` helper to dynamically chain
queries with [drizzle dynamic query
building](https://orm.drizzle.team/docs/dynamic-query-building).
This is more type-safe, more readable and requires less code
This adds support for running multiple job queue tasks in parallel
within the same workflow while preventing conflicts. Previously, this
would have caused the following issues:
- Job log entries get lost - the final job log is incomplete, despite
all tasks having been executed
- Write conflicts in postgres, leading to unique constraint violation
errors
The solution involves handling job log data updates in a way that avoids
overwriting, and ensuring the final update reflects the latest job log
data. Each job log entry now initializes its own ID, so a given job log
entry’s ID remains the same across multiple, parallel task executions.
## Postgres
In Postgres, we need to enable transactions for the
`payload.db.updateJobs` operation; otherwise, two tasks updating the
same job in parallel can conflict. This happens because Postgres handles
array rows by deleting them all, then re-inserting (rather than
upserting). The rows are stored in a separate table, and the following
scenario can occur:
Op 1: deletes all job log rows
Op 2: deletes all job log rows
Op 1: inserts 200 job log rows
Op 2: insert the same 200 job log rows again => `error: “duplicate key
value violates unique constraint "payload_jobs_log_pkey”`
Because transactions were not used, the rows inserted by Op 1
immediately became visible to Op 2, causing the conflict. Enabling
transactions fixes this. In theory, it can still happen if Op 1 commits
before Op 2 starts inserting (due to the read committed isolation
level), but it should occur far less frequently.
Alongside this change, we should consider inserting the rows using an
upsert (update on conflict), which will get rid of this error
completely. That way, if the insertion of Op 1 is visible to Op 2, Op 2
will simply overwrite it, rather than erroring. Individual job entries
are immutable and job entries cannot be deleted, thus this shouldn't
corrupt any data.
## Mongo
In Mongo, the issue is addressed by ensuring that log row deletions
caused due to different log states in concurrent operations are not
merged back to the client job log, and by making sure the final update
includes all job logs.
There is no duplicate key error in Mongo because the array log resides
in the same document and duplicates are simply upserted. We cannot use
transactions in Mongo, as it appears to lock the document in a way that
prevents reliable parallel updates, leading to:
`MongoServerError: WriteConflict error: this operation conflicted with
another operation. Please retry your operation or multi-document
transaction`
You can access the database name from `sanitizedConfig.db.name`. But
currently, it' not possible to access the db name from the unsanitized
config.
Plugins only have access to the unsanitized config. This change allows
db adapters to return the db name early, which will allow plugins to
conditionally initialize db-specific functionality
### What
This PR improves the `getSafeRedirect` utility to improve security
around open redirect handling.
### How
- Normalizes and decodes the redirect path using `decodeURIComponent`
- Catches malformed encodings with a try/catch fallback
- Blocks open redirects
In 3.0, we made the decision to export all types from the main package
export (e.g. `payload/types` => `payload`). This improves type
discoverability by IDEs and simplifies importing types.
This PR does the same for our db adapters, which still have a separate
`/types` subpath export. While those are kept for
backwards-compatibility, we can remove them in 4.0.
Synchronous file system operations such as `readFileSync` block the
event loop, whereas the asynchronous equivalents (like await
`fs.promises.readFile`) do not. This PR replaces certain synchronous fs
calls with their asynchronous counterparts in contexts where async
operations are already in use, improving performance by avoiding event
loop blocking.
Most of the synchronous calls were in our file upload code. Converting
them to async should theoretically free up the event loop and allow
more, other requests to run in parallel without delay
### What?
Fixed client config caching to properly update when switching languages
in the admin UI.
### Why?
Currently, switching languages doesn't fully update the UI because
client config stays cached with previous language translations.
### How?
Created a language-aware caching system that stores separate configs for
each language and only uses cached config when it matches the active
language.
Before:
```typescript
let cachedClientConfig: ClientConfig | null = global._payload_clientConfig
if (!cachedClientConfig) {
cachedClientConfig = global._payload_clientConfig = null
}
export const getClientConfig = cache(
(args: { config: SanitizedConfig; i18n: I18nClient; importMap: ImportMap }): ClientConfig => {
if (cachedClientConfig && !global._payload_doNotCacheClientConfig) {
return cachedClientConfig
}
// ... create new config ...
}
);
```
After:
```typescript
let cachedClientConfigs: Record<string, ClientConfig> = global._payload_localizedClientConfigs
if (!cachedClientConfigs) {
cachedClientConfigs = global._payload_localizedClientConfigs = {}
}
export const getClientConfig = cache(
(args: { config: SanitizedConfig; i18n: I18nClient; importMap: ImportMap }): ClientConfig => {
const { config, i18n, importMap } = args
const currentLocale = i18n.language
if (!global._payload_doNotCacheClientConfig && cachedClientConfigs[currentLocale]) {
return cachedClientConfigs[currentLocale]
}
// ... create new config with correct translations ...
}
);
```
Also added handling for cache clearing during HMR to ensure
compatibility with the existing system.
Fixes#11406
---------
Co-authored-by: Jacob Fletcher <jacobsfletch@gmail.com>
Replaces the queue pattern used within autosave with the `useQueues`
hook introduced in #11579. To do this, queued tasks now accept an
options object with callbacks which can be used to tie into events of
the process, such as before it begins to prevent it from running, and
after it has finished to perform side effects.
The `useQueues` hook now also maintains an array of queued tasks as
opposed to individual refs.
Lexical nested fields are currently not set-up to handle access control
on the client properly. Despite that, we were passing parent permissions
to `RenderFields`, which causes certain fields to not show up if the
document does not have `create` permission.
In the Cell component for a select field such as our `_status` fields it
will now add a class eg. `selected--published` for the selected option
so it can be easily targeted with CSS.
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
### What?
In the same vein as #11696, this PR optimizes how images are selected
for display in the document edit view. It ensures that only image files
are processed and selects the most appropriate size to minimize
unnecessary downloads and improve performance.
#### Previously:
- Non-image files were being processed unnecessarily, despite not
generating thumbnails.
- Images without a `thumbnailURL` defaulted to their original full size,
even when smaller, optimized versions were available.
#### Now:
- **Only images** are processed for thumbnails, avoiding redundant
requests for non-images.
- **The smallest available image within a target range** (`40px -
180px`) is prioritized for display.
- **If no images fit within this range**, the logic selects:
- The next smallest larger image (if available).
- The **original** image if it is smaller than the next available larger
size.
- The largest **smaller** image if no better fit exists.
### Why?
Prevents unnecessary downloads of non-image files, reduces bandwidth
usage by selecting more efficient image sizes and improves load times
and performance in the edit view.
### How?
- **Filters out non-image files** when determining which assets to
display.
- Uses the same algorithm as in #11696 but turns it into a reusable
function to be used in various areas around the codebase. Namely the
upload field hasOne and hasMany components.
Before (4.5mb transfer):
After (15.9kb transfer):
There are cases when a storage plugin is disabled during development and
enabled in production. This will result in import maps that differ
depending on if they're generated during development or production.
In a lot of cases, those import maps are generated during
development-only. During production, we just re-use what was generated
locally. This will cause missing import map entries for those plugins
that are disabled during development.
This PR ensures the import map entries are added regardless of the
enabled state of those plugins. This is necessary for our
generate-templates script to not omit the vercel blob storage import map
entry.
When selecting query presets from the list drawer, all query presets are
available for selection, even if unrelated to the underlying collection.
When selecting one of these presets, the list view will crash with
client-side exceptions because the columns and filters that are applied
are incompatible.
The fix is to the thread `filterOptions` through the query presets
drawer. This will ensure that only related collections are shown.
When rendering custom fields nested within arrays or blocks, such as the
Lexical rich text editor which is treated as a custom field, these
fields will sometimes disappear when form state requests are invoked
sequentially. This is especially reproducible on slow networks.
This is because form state invocations are placed into a [task
queue](https://github.com/payloadcms/payload/pull/11579) which aborts
the currently running tasks when a new one arrives. By doing this, local
form state is never dispatched, and the second task in the queue becomes
stale.
The fix is to _not_ abort the currently running task. This will trigger
a complete rendering cycle, and when the second task is invoked, local
state will be up to date.
Fixes#11340, #11425, and #11824.
Continuation of #11489. This adds a new, optional `updateJobs` db
adapter method that reduces the amount of database calls for the jobs
queue.
## MongoDB
### Previous: running a set of 50 queued jobs
- 1x db.find (= 1x `Model.paginate`)
- 50x db.updateOne (= 50x `Model.findOneAndUpdate`)
### Now: running a set of 50 queued jobs
- 1x db.updateJobs (= 1x `Model.find` and 1x `Model.updateMany`)
**=> 51 db round trips before, 2 db round trips after**
### Previous: upon task completion
- 1x db.find (= 1x `Model.paginate`)
- 1x db.updateOne (= 1x `Model.findOneAndUpdate`)
### Now: upon task completion
- 1x db.updateJobs (= 1x `Model.findOneAndUpdate`)
**=> 2 db round trips before, 1 db round trip after**
## Drizzle (e.g. Postgres)
### running a set of 50 queued jobs
- 1x db.query[tablename].findMany
- 50x db.select
- 50x upsertRow
This is unaffected by this PR and will be addressed in a future PR
Within auth-enabled collections, we inject the `password` and
`confirmPassword` fields into the field schema map. While this is fine
within the edit view where these fields are used, this breaks field
paths within the version diff view where unnamed fields are no longer
able to lookup their corresponding config. This is because the presence
of these injected fields increments the field indices by two.
A temporary fix for this is to simply inject these fields _last_ into
the schema map. This way their presence does not disrupt field path
generation. A long term fix should be implemented, however, where these
fields actually exist on the collection config itself. This way no
config mutation would be required as the sanitized config would the
single source of truth.
To do this, we'd need to ensure that these fields do not appear in any
APIs, and that they do not generate types, etc.
This will improve performance when updating a single document in
postgres/drizzle, if the ID is known.
Previously, this resulted in 2 sequential operations:
- `db.select `to fetch the document by the ID
- `upsertRow` to update the document (multiple db operations)
This PR removes the unnecessary `db.select` call, as the document ID is
already known
### What
The `crypto.randomUUID()` function was causing errors in non-secure
contexts (HTTP), as it is only available in secure contexts (HTTPS).
### How
Added a fallback to generate UUIDs using the `uuid` library when
`crypto.randomUUID()` is not available.
Fixes#11825
This PR introduces a new utility function, `getSafeRedirect`, to
sanitize and validate redirect paths used in the login flow.
It replaces the previous use of `encodeURIComponent` and inline string
checks with a centralized, reusable, and more secure approach.
#### `getSafeRedirect` utility:
- Ensures redirect paths start with a single `/`
- Blocks protocol-relative URLs (e.g., `//evil.com`)
- Blocks JavaScript schemes (e.g., `/javascript:alert(1)`)
- Blocks full URL redirects like `/http:` or `/https:`
This ensures that the lexical field can be rendered without having to
wrap it inside an `EntityVisibilityProvider`, making it a bit easier to
manually render the lexical field in a custom component.
same comment as in #11560, #11831, #11226:
> In `src/index.ts` I see four more errors in my IDE that don't appear
when I run the typecheck in the CLI with `tsc --noEmit`.
Fixes#11458
Some complex, nested fields were receiving incorrect field paths and
schema paths, leading to a `"Error: No client field found"` error.
This PR ensures field paths are calculated correctly, by matching it to
how they're calculated in payload hooks.
Query Presets allow you to save and share filters, columns, and sort
orders for your collections. This is useful for reusing common or
complex filtering patterns and column configurations across your team.
Query Presets are defined on the fly by the users of your app, rather
than being hard coded into the Payload Config.
Here's a screen recording demonstrating the general workflow as it
relates to the list view. Query Presets are not exclusive to the admin
panel, however, as they could be useful in a number of other contexts
and environments.
https://github.com/user-attachments/assets/1fe1155e-ae78-4f59-9138-af352762a1d5
Each Query Preset is saved as a new record in the database under the
`payload-query-presets` collection. This will effectively make them
CRUDable and allows for an endless number of preset configurations. As
you make changes to filters, columns, limit, etc. you can choose to save
them as a new record and optionally share them with others.
Normal document-level access control will determine who can read,
update, and delete these records. Payload provides a set of sensible
defaults here, such as "only me", "everyone", and "specific users", but
you can also extend your own set of access rules on top of this, such as
"by role", etc. Access control is customizable at the operation-level,
for example you can set this to "everyone" can read, but "only me" can
update.
To enable the Query Presets within a particular collection, set
`enableQueryPresets` on that collection's config.
Here's an example:
```ts
{
// ...
enableQueryPresets: true
}
```
Once enabled, a new set of controls will appear within the list view of
the admin panel. This is where you can select and manage query presets.
General settings for Query Presets are configured under the root
`queryPresets` property. This is where you can customize the labels,
apply custom access control rules, etc.
Here's an example of how you might augment the access control properties
with your own custom rule to achieve RBAC:
```ts
{
// ...
queryPresets: {
constraints: {
read: [
{
label: 'Specific Roles',
value: 'specificRoles',
fields: [roles],
access: ({ req: { user } }) => ({
'access.update.roles': {
in: [user?.roles],
},
}),
},
],
}
}
}
```
Related: #4193 and #3092
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
Previously, if you were querying a collection that has a join field with
`draft: true`, and the join field's collection also has
`versions.drafts: true` our db adapter would still query the original
SQL table / mongodb collection instead of the versions one which isn't
quite right since we respect `draft: true` when populating relationships
**BREAKING CHANGE:**
This bumps the **minimum required Next.js** version from 15.0.0 to
15.2.3. This update is necessary due to a critical security
vulnerability found in earlier Next.js versions, which requires an
exception to our standard semantic versioning process.
Additionally, this bumps all templates to the latest Next.js and Payload
versions.
This is already supported in Postgres / SQLite.
For example:
```
const result = await payload.find({
collection: 'directors',
depth: 0,
sort: '-movies.name', // movies is a relationship field here
})
```
Removes the condition in tests:
```
// no support for sort by relation in mongodb
if (isMongoose(payload)) {
return
}
```
This PR adds support for passing additional props to the HTML element of
Next.js `RootLayout`.
#### Context
In our setup, we use several custom Chakra UI components. This change
enables us to add a custom font `className` to the HTML element,
following the official Chakra UI documentation:
[Using custom fonts in Chakra UI with
Next.js](https://v2.chakra-ui.com/getting-started/nextjs-app-guide#using-custom-font)
#### Example Usage
With this update, we can now pass a `className` for custom fonts like
this:
```tsx
import { Rubik } from 'next/font/google'
const rubik = Rubik({
subsets: ['latin'],
variable: '--font-rubik',
})
const Layout = ({ children }: Args) => {
return (
<RootLayout htmlProps={{ className: rubik.variable }}>
{children}
</RootLayout>
);
}
```
Passes the `i18n` arg through field label and description functions.
This is to avoid using custom components when simply needing to
translate a `StaticLabel` object, such as collection labels.
Here's an example:
```ts
{
labels: {
singular: {
en: 'My Collection'
}
},
fields: [
// ...
{
type: 'collapsible',
label: ({ i18n }) => `Translate this: ${getTranslation(collectionConfig.labels.singular, i18n)}`
// ...
}
]
}
```
Previously, our job queue system relied on `payload.*` operations, which
ran very frequently:
- whenever job execution starts, as all jobs need to be set to
`processing: true`
- every single time a task completes or fails, as the job log needs to
be updated
- whenever job execution stops, to mark it as completed and to delete it
(if `deleteJobOnComplete` is set)
This PR replaces these with direct `payload.db.*` calls, which are
significantly faster than payload operations. Given how often the job
queue system communicates with the database, this should be a massive
performance improvement.
## How it affects running hooks
To generate the task status, we previously used an `afterRead` hook.
Since direct db adapter calls no longer execute hooks, this PR
introduces new `updateJob` and `updateJobs` helpers to handle task
status generation outside the normal payload hook lifecycle.
Additionally, a new `runHooks` property has been added to the global job
configuration. While setting this to `true` can be useful if custom
hooks were added to the `payload-jobs` collection config, this will
revert the job system to use normal payload operations.
This should be avoided as it degrades performance. In most cases, the
`onSuccess` or `onFail` properties in the job config will be sufficient
and much faster.
Furthermore, if the `depth` property is set in the global job
configuration, the job queue system will also fall back to the slower,
normal payload operations.
---------
Co-authored-by: Dan Ribbens <DanRibbens@users.noreply.github.com>
