Adds new plugin-import-export initial version.
Allows for direct download and creation of downloadable collection data
stored to a json or csv uses the access control of the user creating the
request to make the file.
config options:
```ts
/**
* Collections to include the Import/Export controls in
* Defaults to all collections
*/
collections?: string[]
/**
* Enable to force the export to run synchronously
*/
disableJobsQueue?: boolean
/**
* This function takes the default export collection configured in the plugin and allows you to override it by modifying and returning it
* @param collection
* @returns collection
*/
overrideExportCollection?: (collection: CollectionOverride) => CollectionOverride
// payload.config.ts:
plugins: [
importExportPlugin({
collections: ['pages', 'users'],
overrideExportCollection: (collection) => {
collection.admin.group = 'System'
collection.upload.staticDir = path.resolve(dirname, 'uploads')
return collection
},
disableJobsQueue: true,
}),
],
```
---------
Co-authored-by: Jessica Chowdhury <jessica@trbl.design>
Co-authored-by: Kendell Joseph <kendelljoseph@gmail.com>
Deep‐merging the collection config defaults during sanitization causes all collection fields to end up with different object references. This is not only slow, but can also lead to unpredictable behavior: mutations made before collection sanitization are reflected in the field config, while mutations made afterward, using the same object reference, are not reflected in the collection’s field config.
Specifically, the following happened:
1. A Block was defined in the module scope.
2. It was then added to both a collection’s blocks field and the config.blocks property.
3. Rich text sanitization promises for config.blocks were collected.
4. The collection config was sanitized.
5. The config.blocks sanitization promises were awaited.
6. Rich text fields were sanitized in config.blocks, but ended up not being sanitized in the collection config referencing the same block, because the object reference held by the promise callback no longer matched the collection config’s object reference. The collection config block did not create its own rich text sanitization promise, as `_sanitized: true` was set on the block during the earlier config.blocks sanitization, which skipped it.
Our config defaults pattern was brittle in general. It’s easy to misuse object spreading or to mutate the config defaults later when you intended only to mutate the payload or collection config. Our current approach was vulnerable to this because it retained some object references from the config defaults.
This PR introduces reliable merge functions that are faster and ensure no object references are shared with defaults that reside in the module scope.
### What?
Supersedes https://github.com/payloadcms/payload/pull/11490.
Refactors imports of `formatAdminURL` to import from `payload/shared`
instead of `@payloadcms/ui/shared`. The ui package now imports and
re-exports the function to prevent this from being a breaking change.
### Why?
This makes it easier for other packages/plugins to consume the
`formatAdminURL` function instead of needing to implement their own or
rely on the ui package for the utility.
When uploading file via client side upload we invalidate it then on the
server side with re-uploading. This works fine with most adapters since
they just replace the old file under the same key. UploadThing works
differently and generates a new key every time.
Example of the issue:
<img width="611" alt="image"
src="https://github.com/user-attachments/assets/9c01b52a-d159-4f32-9f66-3b5fbadab7b4"
/>
Now, we clear the old file before doing re-upload.
Previously, AVIF images were not converted to other file types as
expected, despite `upload.formatOptions` specifying a different file
type.
The issue was due to `canResizeImage` not recognizing `'image/avif',`
causing `fileSupportsResize` to return `false` and preventing the image
from undergoing format conversion.
This fix updates `canResizeImage` to include `'image/avif'`, ensuring
that AVIF images are processed correctly and converted to a different
file type when specified in `formatOptions`.
Fixes#10694Fixes#9985
This PR adds a new `limit` property to `payload.db.updateMany`. This functionality is required for [migrating our job system to use faster, direct db adapter calls](https://github.com/payloadcms/payload/pull/11489)
When the join field is used, Payload now automatically adds an index on
the target relationship field.
For example:
```
{
name: 'relatedPosts',
type: 'join',
collection: "posts",
on: 'category',
},
{
name: 'category',
type: 'relationship',
relationTo: "categories",
},
```
Here, `index: true` implicitly added to the `category` relationship
field during sanitization to improve querying performance.
The `req.url` property at the page level was not reflective of the
actual URL on localhost. This was because we were passing an
incompatible `url` override into `createLocalReq` (lacking protocol).
This would silently fail to construct the URL object, ultimately losing
the top-level domain on `req.url` as well as the port on `req.origin`
(see #11454).
Closes#11448.
This adds new `payload.jobs.cancel` and `payload.jobs.cancelByID` methods that allow you to cancel already-running jobs, or prevent queued jobs from running.
While it's not possible to cancel a function mid-execution, this will stop job execution the next time the job makes a request to the db, which happens after every task.
The `req.origin` property on the `PayloadRequest` object does not
include the port when running on localhost, a requirement of the [HTML
Living Standard](https://html.spec.whatwg.org/#origin). This was because
we were initializing the url with a fallback of `http://localhost` (no
port). When constructed via `new URL()`, the port is unable to be
extracted. This is fixed by using the `host` property off the headers
object, if it exists, which includes the port.
Partial fix for #11448.
Fixes https://github.com/payloadcms/payload/issues/9767
We allow failing a job queue task by returning `{ state: 'failed' }` from the task, instead of throwing an error. However, previously, this threw an error when trying to update the task in the database. Additionally, it was not possible to customize the error message.
This PR fixes that by letting you return `errorMessage` alongside `{ state: 'failed' }`, and by ensuring the error is transformed into proper json before saving it to the `error` column.
Maintains column state in the URL. This makes it possible to share
direct links to the list view in a specific column order or active
column state, similar to the behavior of filters. This also makes it
possible to change both the filters and columns in the same rendering
cycle, a requirement of the "list presets" feature being worked on here:
#11330.
For example:
```
?columns=%5B"title"%2C"content"%2C"-updatedAt"%2C"createdAt"%2C"id"%5D
```
The `-` prefix denotes that the column is inactive.
This strategy performs a single round trip to the server, ultimately
simplifying the table columns provider as it no longer needs to request
a newly rendered table for itself. Without this change, column state
would need to be replaced first, followed by a change to the filters.
This would make an unnecessary number of requests to the server and
briefly render the UI in a stale state.
This all happens behind an optimistic update, where the state of the
columns is immediately reflected in the UI while the request takes place
in the background.
Technically speaking, an additional database query in performed compared
to the old strategy, whereas before we'd send the data through the
request to avoid this. But this is a necessary tradeoff and doesn't have
huge performance implications. One could argue that this is actually a
good thing, as the data might have changed in the background which would
not have been reflected in the result otherwise.
This PR adds a new `returning` option to various db adapter methods. Setting it to `false` where the return value is not used will lead to performance gains, as we don't have to do additional db calls to fetch the updated document and then sanitize it.
### What?
`value` within the beforeValidate field hook was not correctly falling
back to the document value when no value was passed inside the request
for the field.
### Why?
The fallback logic was running after the beforeValidate field hooks are
called.
### How?
Run the fallback logic before running the beforeValidate field hooks.
Fixes https://github.com/payloadcms/payload/issues/10923
### What?
For the join field query adds ability to specify `count: true`, example:
```ts
const result = await payload.find({
joins: {
'group.relatedPosts': {
sort: '-title',
count: true,
},
},
collection: "categories",
})
result.group?.relatedPosts?.totalDocs // available
```
### Why?
Can be useful to implement full pagination / show total related
documents count in the UI.
### How?
Implements the logic in database adapters. In MongoDB it's additional
`$lookup` that has `$count` in the pipeline. In SQL, it's additional
subquery with `COUNT(*)`. Preserves the current behavior by default,
since counting introduces overhead.
Additionally, fixes a typescript generation error for join fields.
Before, `docs` and `hasNextPage` were marked as nullable, which is not
true, these fields cannot be `null`.
Additionally, fixes threading of `joinQuery` in
`transform/read/traverseFields` for group / tab fields recursive calls.
This reverts commit 69c0d09 in #11390.
In order to future proof column prefs, it probably is best to continue
to use the current shape. This change was intended to ensure that as
little transformation to URL params was made as possible for #11387, but
we will likely transform them after all.
This will ensure that we can add support for additional properties over
time, as needed. For example, if we hypothetically wanted to add a
custom `label` or similar feature to columns prefs, it would make more
sense to use explicit properties to identity `accessor` and `active`.
For example:
```ts
[
{
accessor: "title",
active: true,
label: 'Custom Label' // hypothetical
}
]
```
Previously, behavior with custom IDs and `select` query was incorrect.
By default, the `id` field is guaranteed to be selected, even if it
doesn't exist in the `select` query, this wasn't true for custom IDs.
This PR fixes an issue where bulk upload attempts to generate thumbnails
for non-image files, causing errors on the page.
The fix ensures that thumbnail generation is skipped for non-image
files, preventing unnecessary errors.
Fixes#10428
Transforms how column prefs are stored in the database. This change
reduces the complexity of the `columns` property by removing the
unnecessary `accessor` and `active` keys.
This change is necessary in order to [maintain column state in the
URL](https://github.com/payloadcms/payload/pull/11387), where the state
itself needs to be as concise as possible. Does so in a non-breaking
way, where the old column shape is transformed as needed.
Here's an example:
Before:
```ts
[
{
accessor: "title",
active: true
}
]
```
After:
```ts
[
{
title: true
}
]
```
Continuation of https://github.com/payloadcms/payload/pull/11372 but for our collection, global and auth operations
Previously, we were quite frequently using `.reduce()` to run hooks. This PR replaces them with simple `for` loops, which is less overhead, less code, less confusing and simpler to understand.
Previously, we were quite frequently using `.reduce()` to sequentially run field hooks. This PR replaces them with simple `for` loops, which is less overhead, less code, less confusing and simpler to understand.
Additionally, it refactors `mergeLocaleActions` which previously was unnecessarily complex. They no longer entail async code, thus we no longer have to juggle with promises
Small performance improvement for types generation and anywhere else
`configToJSONSchema` can be used.
We did a deep copy of the whole sanitized entity config, which can be
expensive. Now, to do the needed mutation of `flattenedFields`, we just
create a new reference of `flattenedFields` for that.
`payload.find` queries can be made faster by specifying `limit: 1` and `pagination: false` when only the first document is needed. This PR applies those options to various queries to improve performance.
Previously, the `bin` configuration wasn't working at all.
Possibly because in an ESM environment this cannot work, because
`import` always returns an object with a default export under the
`module` key.
```ts
const script: BinScript = await import(pathToFileURL(userBinScript.scriptPath).toString())
await script(config)
```
Now, this works, but you must define a `script` export from your file.
Attached an integration test that asserts that it actually works. Added
documentation on how to use it, as previously it was missing.
This can be also helpful for plugins.
### Documentation
Using the `bin` configuration property, you can inject your own scripts
to `npx payload`.
Example for `pnpm payload seed`:
Step 1: create `seed.ts` file in the same folder with
`payload.config.ts` with:
```ts
import type { SanitizedConfig } from 'payload'
import payload from 'payload'
// Script must define a "script" function export that accepts the sanitized config
export const script = async (config: SanitizedConfig) => {
await payload.init({ config })
await payload.create({ collection: 'pages', data: { title: 'my title' } })
payload.logger.info('Succesffully seeded!')
process.exit(0)
}
```
Step 2: add the `seed` script to `bin`:
```ts
export default buildConfig({
bin: [
{
scriptPath: path.resolve(dirname, 'seed.ts'),
key: 'seed',
},
],
})
```
Now you can run the script using:
```sh
pnpm payload seed
```
Just like https://github.com/payloadcms/payload/pull/11269, we stop passing through `req` to db operations in `checkDocumentLockStatus`.
After extensive testing, this seems to get rid of all transaction errors that occurred when I was testing autosave against a remote mongo DB.
We keep the `req` for postgres, as it mysteriously breaks in CI - this cannot be reproduced locally
When `/uploads/file/file-path.jpg` endpoint is requested, and
`file-path.jpg` _was found_ in the database, but was not found on the
disk, the error like this is printed to the console:
```
[03:34:54] ERROR: ENOENT: no such file or directory, stat '/Users/sasha/work/payload/test/uploads/collections/Upload1/uploads/Screenshot 2025-02-19 at 18.50.46.png'
err: {
"type": "Error",
"message": "ENOENT: no such file or directory, stat '/Users/sasha/work/payload/test/uploads/collections/Upload1/uploads/Screenshot 2025-02-19 at 18.50.46.png'",
"stack":
Error: ENOENT: no such file or directory, stat '/Users/sasha/work/payload/test/uploads/collections/Upload1/uploads/Screenshot 2025-02-19 at 18.50.46.png'
at async Object.stat (node:internal/fs/promises:1037:18)
at async getFileHandler (webpack-internal:///(rsc)/./packages/payload/src/uploads/endpoints/getFile.ts:59:19)
at async handleEndpoints (webpack-internal:///(rsc)/./packages/payload/src/utilities/handleEndpoints.ts:178:26)
at async eval (webpack-internal:///(rsc)/./packages/next/src/routes/rest/index.ts:27:26)
at async AppRouteRouteModule.do (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/compiled/next-server/app-route.runtime.dev.js:10:32847)
at async AppRouteRouteModule.handle (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/compiled/next-server/app-route.runtime.dev.js:10:39868)
at async doRender (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/base-server.js:1452:42)
at async responseGenerator (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/base-server.js:1822:28)
at async DevServer.renderToResponseWithComponentsImpl (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/base-server.js:1832:28)
at async DevServer.renderPageComponent (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/base-server.js:2259:24)
at async DevServer.renderToResponseImpl (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/base-server.js:2297:32)
at async DevServer.pipeImpl (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/base-server.js:959:25)
at async NextNodeServer.handleCatchallRenderRequest (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/next-server.js:281:17)
at async DevServer.handleRequestImpl (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/base-server.js:853:17)
at async /Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/dev/next-dev-server.js:371:20
at async Span.traceAsyncFn (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/trace/trace.js:153:20)
at async DevServer.handleRequest (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/dev/next-dev-server.js:368:24)
at async invokeRender (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/lib/router-server.js:230:21)
at async handleRequest (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/lib/router-server.js:408:24)
at async NextCustomServer.requestHandlerImpl (/Users/sasha/work/payload/node_modules/.pnpm/next@15.1.5_@opentelemetry+api@1.9.0_@playwright+test@1.50.0_babel-plugin-macros@3.1.0_babel-_vtyh6lbrnb3jbtpvhe5wg2qgzq/node_modules/next/dist/server/lib/router-server.js:432:13)
at async Server.<anonymous> (file:///Users/sasha/work/payload/test/dev.ts:1:2848)
"errno": -2,
"code": "ENOENT",
"syscall": "stat",
"path": "/Users/sasha/work/payload/test/uploads/collections/Upload1/uploads/Screenshot 2025-02-19 at 18.50.46.png"
}
```
Which is quite noisy as we understand why this error happens (and it
might be intentional when you load production DB to local and don't have
any files)
Now, the logging here in case of `ENOENT` is simplified to this:
```
[03:43:35] ERROR: File Screenshot 2025-02-19 at 18.50.46.png for collection uploads-1 is missing on the disk. Expected path: /Users/sasha/work/payload/test/uploads/collections/Upload1/uploads/Screenshot 2025-02-19 at 18.50.46.png
```
Fixes https://github.com/payloadcms/payload/issues/6246
### What?
Adds JSDoc for options of all collection Local API operations
_Every_ property now is documented there, even those that aren't on our
website docs.
### Why?
This is useful to have, now you can hover over any property to see what
it does in your editor.
Some properties also link to the documentation website directly for more
info.
### How?
Updates every collection operation arguments definition with JSDoc.
For globals will be a separate PR.
<!--
Thank you for the PR! Please go through the checklist below and make
sure you've completed all the steps.
Please review the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository if you haven't already.
The following items will ensure that your PR is handled as smoothly as
possible:
- PR Title must follow conventional commits format. For example, `feat:
my new feature`, `fix(plugin-seo): my fix`.
- Minimal description explained as if explained to someone not
immediately familiar with the code.
- Provide before/after screenshots or code diffs if applicable.
- Link any related issues/discussions from GitHub or Discord.
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Fixes #
-->
### What?
This PR introduces support for the `admin.group` property in block
configs. This property enables blocks to be grouped under a common,
potentially localized, label in the block drawer component. This makes
it easier to sort through large collections of blocks. Previously, all
blocks would be in one common layout.
This PR also encompasses documentation changes and e2e tests to check
for the rendering of group labels.
### Why?
To make it easier to organize many blocks in block fields.
### How?
By introducing a new `admin.group` property in block configs and
assembling them in the blocks drawer component.
Before:
After:
Demo:
[Editing---Block-Field---Payload-groups-demo.webm](https://github.com/user-attachments/assets/2b351dc1-0d14-4a5b-ae71-bcd31fbb23df)
Addresses #5609
### What?
Fixes translation errors that are thrown when JSON field validation
outputs an error.
### How?
Removes translation function `t()` from wrapping the errors and adds
translation keys that were missing from `clientKeys.ts`.
Fixes#10543
### What?
Adding `assetPrefix` to the `next.config` prevents the hot module
reloading functionality.
### Why & How?
Need to incorporate `assetPrefix` into the URL generated for webpack
HMR.
Fixes#11150
#### Testing
1. Add `assetPrefix: '/test'` to the `next.config.mjs` in the root
folder
2. Run `pnpm test _community`
3. Go to the `_community/collections/posts` config and change a field
4. Open post collection in browser and see no change (if this PR is
checked out then you _**will**_ see the change)
### What?
The admin panel was not respecting where constraints returned from the
readAccess function.
### Why?
`getEntityPolicies` was always using `find` when looping over the
operations, but `readVersions` should be using `findVersions`.
### How?
When the operation is `readVersions` run the `findVersions` operation.
Fixes https://github.com/payloadcms/payload/issues/11240
Just like https://github.com/payloadcms/payload/pull/11269, we stop
passing through `req` to db operations in `checkDocumentLockStatus`.
After extensive testing, this seems to get rid of all transaction errors
that occurred when I was testing autosave against a remote mongo DB.
In the `findOne` db operation, we return `null` if the document was not
found.
For single-document delete and update operations, if the document you
wanted to update is not found, the following runtime error is thrown
instead: `Cannot read properties of null (reading '_id')`.
This PR correctly handles these cases and returns `null` from the db
method, just like the `findOne` operation.
This feature allows you to specify `collection` for the join field as
array.
This can be useful for example to describe relationship linking like
this:
```ts
{
slug: 'folders',
fields: [
{
type: 'join',
on: 'folder',
collection: ['files', 'documents', 'folders'],
name: 'children',
},
{
type: 'relationship',
relationTo: 'folders',
name: 'folder',
},
],
},
{
slug: 'files',
upload: true,
fields: [
{
type: 'relationship',
relationTo: 'folders',
name: 'folder',
},
],
},
{
slug: 'documents',
fields: [
{
type: 'relationship',
relationTo: 'folders',
name: 'folder',
},
],
},
```
Documents and files can be placed to folders and folders themselves can
be nested to other folders (root folders just have `folder` as `null`).
Output type of `Folder`:
```ts
export interface Folder {
id: string;
children?: {
docs?:
| (
| {
relationTo?: 'files';
value: string | File;
}
| {
relationTo?: 'documents';
value: string | Document;
}
| {
relationTo?: 'folders';
value: string | Folder;
}
)[]
| null;
hasNextPage?: boolean | null;
} | null;
folder?: (string | null) | Folder;
updatedAt: string;
createdAt: string;
}
```
While you could instead have many join fields (for example
`childrenFolders`, `childrenFiles`) etc - this doesn't allow you to
sort/filter and paginate things across many collections, which isn't
trivial. With SQL we use `UNION ALL` query to achieve that.
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
