Follow up to #11375.
When setting `filterOptions` on relationship or upload fields _that are
nested within a named field_, those options won't be applied to the
`Filter` component in the list view.
This is because of how we key the results when resolving `filterOptions`
on the server. Instead of using the field path as expected, we were
using the field name, causing a failed lookup on the front-end. This
also solves an issue where two fields with the same name would override
each other's `filterOptions`, since field names alone are not unique.
Unrelated: this PR also does some general housekeeping to e2e test
helpers.
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1211332845301583
### What?
Added support for `disableListColumn` and `disableListFilter` admin
properties on imageSize configurations that automatically apply to all
fields within the corresponding size group.
### Why?
Upload collections with multiple image sizes can clutter the admin list
view with many size-specific columns and filters. This feature allows
developers to selectively hide size fields from list views while keeping
them accessible in the document edit view.
### How?
Modified `getBaseFields.ts` to inherit admin properties from imageSize
configuration and apply them to all nested fields (url, width, height,
mimeType, filesize, filename) within each size group. The implementation
uses conditional spread operators to only apply these properties when
explicitly set to `true`, maintaining backward compatibility.
Adds a new property to the config that enables the Select API in the
list view. This is a performance opt-in, where only the active columns
(and those specified in `forceSelect`) are queried. This can greatly
improve performance, especially for collections with large documents or
many fields.
To enable this, use the `admin.enableListViewSelectAPI` in your
Collection Config:
```ts
import type { CollectionConfig } from 'payload'
export const Posts: CollectionConfig = {
// ...
admin: {
enableListViewSelectAPI: true // This will select only the active columns (and any `forceSelect` fields)
}
}
```
Note: The `enableListViewSelectAPI` property is currently labeled as
experimental, as it will likely become the default behavior in v4 and be
deprecated. The reason it cannot be the default now is because cells or
other components may be relying on fully populated data, which will no
longer be the case when using `select`.
For example, if your component relies on a "title" field, this field
will _**not**_ exist if the column is **_inactive_**:
```ts
import type { CollectionConfig } from 'payload'
export const Posts: CollectionConfig = {
// ...
fields: [
// ...
{
name: 'myField',
type: 'text',
hooks: {
afterRead: [
({ doc }) => doc.title // `title` will only be populated if the column is active
]
}
}
]
}
```
There are other cases that might be affected by this change as well, for
example any components relying on the `data` object returned by the
`useListQuery()` hook:
```ts
'use client'
export const MyComponent = () => {
const { data } = useListQuery() // `data.docs` will only contain fields that are selected
// ...
}
```
To ensure title is always present, you will need to add that field to
the `forceSelect` property in your Collection Config:
```ts
import type { CollectionConfig } from 'payload'
export const Posts: CollectionConfig = {
// ...
forceSelect: {
title: true
}
}
```
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1211248751470559
Deflakes the uploads e2e suite. The CORS test was not waiting not long
enough for the response and ultimately fails.
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1211295974585601
Currently, if you don't have delete access to the document, the UI
doesn't allow you to replace the file, which isn't expected. This is
also a UI only restriction, and the API allows you do this fine.
This PR makes so the "remove file" button renders even if you don't have
delete access, while still ensures you have update access.
---------
Co-authored-by: Paul Popus <paul@payloadcms.com>
### What
- filters cookies with the `payload-` prefix in `getExternalFile` by
default (if `externalFileHeaderFilter` is not used).
- Document in `externalFileHeaderFilter`, that the user should handle
the removing of the payload cookie.
### Why
In the Payload application, the `getExternalFile` function sends the
user's cookies to an external server when fetching media, inadvertently
exposing the user's session to that third-party service.
```ts
const headers = uploadConfig.externalFileHeaderFilter
? uploadConfig.externalFileHeaderFilter(Object.fromEntries(new Headers(req.headers)))
: { cookie: req.headers?.get('cookie') };
const res = await fetch(fileURL, {
credentials: 'include',
headers,
method: 'GET',
});
```
Although the
[externalFileHeaderFilter](https://payloadcms.com/docs/upload/overview#collection-upload-options)
function can strip sensitive cookies from the request, the default
config includes the session cookie, violating the secure-by-default
principle.
### How
- If `externalFileHeaderFilter` is not defined, any cookie beginning
with `payload-` is filtered.
- Added 2 tests: both for the case where `externalFileHeaderFilter` is
defined and for the case where it is not.
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1210561338171125
### What
Introduces an additional `mimeType` validation based on the actual file
data to ensure the uploaded file matches the allowed `mimeTypes` defined
in the upload config.
### Why?
The current validation relies on the file extension, which can be easily
manipulated. For example, if only PDFs are allowed, a JPEG renamed to
`image.pdf` would bypass the check and be accepted. This change prevents
such cases by verifying the true MIME type.
### How?
Performs a secondary validation using the file’s binary data (buffer),
providing a more reliable MIME type check.
Fixes#12905
Previously, we were performing this check before calling the fetch
function. This changes it to perform the check within the dispatcher.
It adjusts the int tests to both trigger the dispatcher lookup function
(which is only triggered when not already passing a valid IP) and the
check before calling fetch
---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
- https://app.asana.com/0/0/1210733180484570
Adds `restrictedFileTypes` (default: `false`) to upload collections
which prevents files on a restricted list from being uploaded.
To skip this check:
- set `[Collection].upload.restrictedFileTypes` to `true`
- set `[Collection].upload.mimeType` to any type(s)
### What?
The "Preview Sizes" button in the file upload UI was not showing up if:
- `crop` and `focalPoint` were both `false`
- No `customUploadActions` were provided
- But image sizes were configured
### Why?
This happened because `UploadActions` wasn’t rendered at all unless
adjustments or custom actions were present.
### How?
Update the conditional in `StaticFileDetails` to also render
`UploadActions` when:
- `hasImageSizes` is `true` and the document has a `filename`
Fixes#12832
### What?
Adds `constructorOptions` property to the upload config to allow any of
[these options](https://sharp.pixelplumbing.com/api-constructor/) to be
passed to the Sharp library.
### Why?
Users should be able to extend the Sharp library config as needed, to
define useful properties like `limitInputPixels` etc.
### How?
Creates new config option `constructorOptions` which passes any
compatible options directly to the Sharp library.
#### Reported by client.
Filters URLs to avoid issues with SSRF
Had to use `undici` instead of native `fetch` because it was the only
viable alternative that supported both overriding agent/dispatch and
also implemented `credentials: include`.
[More info
here.](https://blog.doyensec.com/2023/03/16/ssrf-remediation-bypass.html)
---------
Co-authored-by: Elliot DeNolf <denolfe@gmail.com>
This clarifies that jobs.autoRun only *runs* already-queued jobs. It does not queue the jobs for you.
Also adds an e2e test as this functionality had no e2e coverage
### What?
The browser was incorrectly setting the mimetype for `.glb` and `.gltf`
files to `application/octet-stream` when uploading when they should be
receiving proper types consistent with `glb` and `gltf`.
This patch adds logic to infer the correct `MIME` type for `.glb` files
(`model/gltf-binary`) & `gltf` files (`model/gltf+json`) based on file
extension during multipart processing, ensuring consistent MIME type
detection regardless of browser behavior.
Fixes#12620
This PR updates `generateFileData` to skip applying `resizeOptions`
after updating an image if `resizeOptions.withoutEnlargement` is `true`
and the original image size is smaller than the dimensions defined in
`resizeOptions`.
This prevents unintended re-resizing of already resized images when
updating or modifying metadata without uploading a new file.
This change ensures that:
- Resizing is skipped if withoutEnlargement: true
- Resizing still occurs if withoutEnlargement: false or unset
This resolves an issue where images were being resized again
unnecessarily when updating an upload.
Fixes#12280
I think it's easier to review this PR commit by commit, so I'll explain
it this way:
## Commits
1. [parallelize eslint script (still showing logs results in
serial)](c9ac49c12d):
Previously, `--concurrency 1` was added to the script to make the logs
more readable. However, turborepo has an option specifically for these
use cases: `--log-order=grouped` runs the tasks in parallel but outputs
them serially. As a result, the lint script is now significantly faster.
2. [run pnpm
lint:fix](9c128c276a)
The auto-fix was run, which resolved some eslint errors that were
slipped in due to the use of `no-verify`. Most of these were
`perfectionist` fixes (property ordering) and the removal of unnecessary
assertions. Starting with this PR, this won't happen again in the
future, as we'll be verifying the linter in every PR across the entire
codebase (see commit 7).
3. [fix eslint non-autofixable
errors](700f412a33)
All manual errors have been resolved except for the configuration errors
addressed in commit 5. Most were React compiler violations, which have
been disabled and commented out "TODO" for now. There's also an unused
`use no memo` and a couple of `require` errors.
4. [move react-compiler linter to eslint-config
package](4f7cb4d63a)
To simplify the eslint configuration. My concern was that there would be
a performance regression when used in non-react related packages, but
none was experienced. This is probably because it only runs on .tsx
files.
5. [remove redundant eslint config files and fix
allowDefaultProject](a94347995a)
The main feature introduced by `typescript-eslint` v8 was
`projectService`, which automatically searches each file for the closest
`tsconfig`, greatly simplifying configuration in monorepos
([source](https://typescript-eslint.io/blog/announcing-typescript-eslint-v8#project-service)).
Once I moved `projectService` to `packages/eslint-config`, all the other
configuration files could be easily removed.
I confirmed that pnpm lint still works on individual packages.
The other important change was that the pending eslint errors from
commits 2 and 3 were resolved. That is, some files were giving the
error: "[File] was not found by the project service. Consider either
including it in the tsconfig.json or including it in
allowDefaultProject." Below I copy the explanatory comment I left in the
code:
```ts
// This is necessary because `tsconfig.base.json` defines `"rootDir": "${configDir}/src"`,
// And the following files aren't in src because they aren't transpiled.
// This is typescript-eslint's way of adding files that aren't included in tsconfig.
// See: https://typescript-eslint.io/troubleshooting/typed-linting/#i-get-errors-telling-me--was-not-found-by-the-project-service-consider-either-including-it-in-the-tsconfigjson-or-including-it-in-allowdefaultproject
// The best practice is to have a tsconfig.json that covers ALL files and is used for
// typechecking (with noEmit), and a `tsconfig.build.json` that is used for the build
// (or alternatively, swc, tsup or tsdown). That's what we should ideally do, in which case
// this hardcoded list wouldn't be necessary. Note that these files don't currently go
// through ts, only through eslint.
```
6. [Differentiate errors from warnings in VScode ESLint
Rules](5914d2f48d)
There's no reason to do that. If an eslint rule isn't an error, it
should be disabled or converted to a warning.
7. [Disable skip lint, and lint over the entire repo now that it's
faster](e4b28f1360)
The GitHub action linted only the files that had changed in the PR.
While this seems like a good idea, once exceptions were introduced with
[skip lint], they opened the door to propagating more and more errors.
Often, the linter was skipped, not because someone introduced new
errors, but because they were trying to avoid those that had already
crept in, sometimes accidentally introducing new ones.
On the other hand, `pnpm lint` now runs in parallel (commit 1), so it's
not that slow. Additionally, it runs in parallel with other GitHub
actions like e2e tests, which take much longer, so it can't represent a
bottleneck in CI.
8. [fix lint in next
package](4506595f91)
Small fix missing from commit 5
9. [Merge remote-tracking branch 'origin/main' into
fix-eslint](563d4909c1)
10. [add again eslint.config.js in payload
package](78f6ffcae7)
The comment in the code explains it. Basically, after the merge from
main, the payload package runs out of memory when linting, probably
because it grew in recent PRs. That package will sooner or later
collapse for our tooling, so we may have to split it. It's already too
big.
## Future Actions
- Resolve React compiler violations, as mentioned in commit 3.
- Decouple the `tsconfig` used for typechecking and build across the
entire monorepo (as explained in point 5) to ensure ts coverage even for
files that aren't transpiled (such as scripts).
- Remove the few remaining `eslint.config.js`. I had to leave the
`richtext-lexical` and `next` ones for now. They could be moved to the
root config and scoped to their packages, as we do for example with
`templates/vercel-postgres/**`. However, I couldn't get it to work, I
don't know why.
- Make eslint in the test folder usable. Not only are we not linting
`test` in CI, but now the `pnpm eslint .` command is so large that my
computer freezes. If each suite were its own package, this would be
solved, and dynamic codegen + git hooks to modify tsconfig.base.json
wouldn't be necessary
([related](https://github.com/payloadcms/payload/pull/11984)).
### What?
In the List View, row data related to images and relationships gets
stuck when you go from one page to another.
### Why?
The `key` we are providing is not unique and not triggering the DOM to
update.
### How?
Uses the `row id` as a unique key prop to each table row to ensure
proper re-rendering of rows during pagination.
#### Testing
Adds e2e test to `upload` test suite. You can recreate the issue using
the `upload` test suite and new `list view preview` collection.
### What?
In the same vein as #11696, this PR optimizes how images are selected
for display in the document edit view. It ensures that only image files
are processed and selects the most appropriate size to minimize
unnecessary downloads and improve performance.
#### Previously:
- Non-image files were being processed unnecessarily, despite not
generating thumbnails.
- Images without a `thumbnailURL` defaulted to their original full size,
even when smaller, optimized versions were available.
#### Now:
- **Only images** are processed for thumbnails, avoiding redundant
requests for non-images.
- **The smallest available image within a target range** (`40px -
180px`) is prioritized for display.
- **If no images fit within this range**, the logic selects:
- The next smallest larger image (if available).
- The **original** image if it is smaller than the next available larger
size.
- The largest **smaller** image if no better fit exists.
### Why?
Prevents unnecessary downloads of non-image files, reduces bandwidth
usage by selecting more efficient image sizes and improves load times
and performance in the edit view.
### How?
- **Filters out non-image files** when determining which assets to
display.
- Uses the same algorithm as in #11696 but turns it into a reusable
function to be used in various areas around the codebase. Namely the
upload field hasOne and hasMany components.
Before (4.5mb transfer):
After (15.9kb transfer):
### What?
This PR ensures that bulk uploads fail if any file is missing, rather
than skipping missing files and proceeding with the upload.
### Why?
This fixes unintended behavior where missing files were skipped,
allowing partial uploads when they shouldn't be allowed.
### How?
- Prevents submission if any file is missing by checking `req.status ===
400`.
- Updates `FileSidebar` to correctly handle cases where a file is
`null`.
Bulk edit can now request a partial form state thanks to #11689. This
means that we only need to build form state (and send it through the
network) for the currently selected fields, as opposed to the entire
field schema.
Not only this, but there is no longer a need to filter out unselected
fields before submitting the form, as the form state will only ever
include the currently selected fields. This is unnecessary processing
and causes an excessive amount of rendering, especially since we were
dispatching actions within a `for` loop to remove each field. React may
have batched these updates, but is bad practice regardless.
Related: stripping unselected fields was also error prone. This is
because the `overrides` function we were using to do this receives
`FormState` (shallow) as an argument, but was being treated as `Data`
(not shallow, what the create and update operations expect).
E.g. `{ myGroup.myTitle: { value: 'myValue' }}` → `{ myGroup: { myTitle:
'myValue' }}`.
This led to the `sanitizeUnselectedFields` function improperly
formatting data sent to the server and would throw an API error upon
submission. This is only evident when sanitizing nested fields. Instead
of converting this data _again_, the select API takes care of this by
ensuring only selected fields exist in form state.
Related: bulk upload was not hitting form state on change. This means
that no field-level validation was occurring on type.
This PR fixes an issue where the Sharp `.resize()` function would round
down an auto-scaled dimension when `fastShrinkOnLoad` was enabled
(enabled by default).
This caused slight discrepancies in height calculations in certain edge
cases.
Be default (`fastShrinkOnLoad: true`), Sharp:
- Uses the built-in shrink-on-load feature for JPEG and WebP
- It is an optimization that prioritizes speed over precision when
resizing images
By setting `fastShrinkOnLoad: false`, we force Sharp to:
- Perform a more accurate resize operation instead of relying on quick
pre-shrink methods.
### Before / Context:
- Upload an image with original dimensions of 1500 × 735
- Define an `imageSize` of the following:
```
{
name: 'thumbnail',
width: 300,
},
```
#### Calculation:
`originalAspectRatio = 1500 / 735 ≈ 2.04081632653`
`resizeHeight = 300 / 2.04081632653`
`resizeHeight = 147`
However, Sharp's `.resize()` calculation would output:
`resizeHeight = 146`
This lead to an error of:
```
[17:05:13] ERROR: extract_area: bad extract area
err: {
"type": "Error",
"message": "extract_area: bad extract area",
"stack":
Error: extract_area: bad extract area
}
```
### After:
Sharp's `.resize()` calculation now correctly outputs:
`resizeHeight = 147`
### What?
We have the option to set `displayPreview: true || false` on upload
collections / upload fields - with the **field** option taking
precedence.
Currently, `displayPreview` is only affecting the list view for the
**_related_** collection.
i.e. if you go to a collection that has an upload field - the preview
will be hidden/shown correctly according to the `displayPreview` option.
<img width="620" alt="Screenshot 2025-03-03 at 12 38 18 PM"
src="https://github.com/user-attachments/assets/c11c2a84-0f64-4a08-940e-8c3f9096484b"
/>
However, when you go directly to the upload collection and look at the
list view - the preview is always shown, not affected by the
`displayPreview` option.
<img width="446" alt="Screenshot 2025-03-03 at 12 39 24 PM"
src="https://github.com/user-attachments/assets/f5e1267a-d98a-4c8c-8d54-93dea6cd2e31"
/>
Also, we have previews within the file field itself - also not being
affected by the `displayPreview` option.
<img width="528" alt="Screenshot 2025-03-03 at 12 40 06 PM"
src="https://github.com/user-attachments/assets/3dd04c9a-3d9f-4823-90f8-b538f3d420f9"
/>
All the upload related previews (excluding preview sizes and upload
editing options) should be affected by the `displayPreview` option.
### How?
Checks for `collection.displayPreview` and `field.displayPreview` in all
places where previews are displayed.
Closes#11404
Fixes https://github.com/payloadcms/payload/issues/11568
### What? Out of sync errors states
- Collaspibles & Tabs were not reporting accurate child error counts
- Arrays could get into a state where they would not update their error
states
- Slight issue with toasts
### Tabs & Collapsibles
The logic for determining matching field paths was not functioning as
intended. Fields were attempting to match with paths such as `_index-0`
which will not work.
### Arrays
The form state was not updating when the server sent back errorPaths.
This PR adds `errorPaths` to `serverPropsToAccept`.
### Toasts
Some toasts could report errors in the form of `my > > error`. This
ensures they will be `my > error`
### Misc
Removes 2 files that were not in use:
- `getFieldStateFromPaths.ts`
- `getNestedFieldState.ts`
Previously, AVIF images were not converted to other file types as
expected, despite `upload.formatOptions` specifying a different file
type.
The issue was due to `canResizeImage` not recognizing `'image/avif',`
causing `fileSupportsResize` to return `false` and preventing the image
from undergoing format conversion.
This fix updates `canResizeImage` to include `'image/avif'`, ensuring
that AVIF images are processed correctly and converted to a different
file type when specified in `formatOptions`.
Fixes#10694Fixes#9985
Maintains column state in the URL. This makes it possible to share
direct links to the list view in a specific column order or active
column state, similar to the behavior of filters. This also makes it
possible to change both the filters and columns in the same rendering
cycle, a requirement of the "list presets" feature being worked on here:
#11330.
For example:
```
?columns=%5B"title"%2C"content"%2C"-updatedAt"%2C"createdAt"%2C"id"%5D
```
The `-` prefix denotes that the column is inactive.
This strategy performs a single round trip to the server, ultimately
simplifying the table columns provider as it no longer needs to request
a newly rendered table for itself. Without this change, column state
would need to be replaced first, followed by a change to the filters.
This would make an unnecessary number of requests to the server and
briefly render the UI in a stale state.
This all happens behind an optimistic update, where the state of the
columns is immediately reflected in the UI while the request takes place
in the background.
Technically speaking, an additional database query in performed compared
to the old strategy, whereas before we'd send the data through the
request to avoid this. But this is a necessary tradeoff and doesn't have
huge performance implications. One could argue that this is actually a
good thing, as the data might have changed in the background which would
not have been reflected in the result otherwise.
Removes all unnecessary `page.waitForURL` methods within e2e tests.
These are unneeded when following a `page.goto` call because the
subsequent page load is already being awaited.
It is only a requirement when:
- Clicking a link and expecting navigation
- Expecting a redirect after a route change
- Waiting for a change in search params
This PR fixes an issue where bulk upload attempts to generate thumbnails
for non-image files, causing errors on the page.
The fix ensures that thumbnail generation is skipped for non-image
files, preventing unnecessary errors.
Fixes#10428
### What?
Two new configuration properties added for upload enabled collections.
- *hideFileInputOnCreate* - Set to `true` to prevent the admin UI from
showing file inputs during document creation, useful for programmatic
file generation.
- *hideRemoveFile* - Set to `true` to prevent the admin UI having a way
to remove an existing file while editing.
### Why?
When using file uploads that get created programmatically in
`beforeOperation` hooks or files created using `jobs`, or when
`filesRequiredOnCreate` is false, you may want to use these new flags to
prevent users from interacting with these controls.
### How?
The new properties only impact the admin UI components to dial in the UX
for various use cases.
Screenshot showing that the upload controls are not available on create:
Screenshot showing hideRemoveFile has removed the ability to remove the
existing file:
Prerequisite for https://github.com/payloadcms/payload/pull/10795
### What?
URL encodes the imageCacheTag query param used to render Media on the
Admin Dashboard
### Why?
The format of the timestamp used as the `imageCacheTag` is causing an
`InvalidQueryStringException` when hosting with Cloudfront + Lambda
(SST/OpenNext)
[See issue](https://github.com/payloadcms/payload/issues/11163)
### How?
Uses `encodeURIComponent` on instances where the `imageCacheTag` is
being formatted for the request URL. (In EditUpload, Thumbnail, and
PreviewSizes)
Fixes#11163
---------
Co-authored-by: Jarrod Flesch <jarrodmflesch@gmail.com>
Previously, data created by other tests was also leaking into unrelated tests, causing them to fail. The new reset-db-between-tests logic added by this PR fixes this.
Additionally, this increases playwright timeouts for CI, and adds a specific timeout override for opening a drawer, as it was incredibly slow in CI
The `fields-relationship` test suite is disorganized to the point of
being unusable. This makes it very difficult to digest at a high level
and add new tests.
This PR cleans it up in the following ways:
- Moves collection configs to their own standalone files
- Moves the seed function to its own file
- Consolidates collection slugs in their own file
- Uses generated types instead of defining them statically
- Wraps the `filterOptions` e2e tests within a describe block
Related, there are three distinct test suites where we manage
relationships: `relationships`, `fields-relationship`, and `fields >
relationships`. In the future we ought to consolidate at least two of
these. IMO the `fields > relationship` suite should remain in place for
general _component level_ UI tests for the field itself, whereas the
other suite could run the integration tests and test the more complex UI
patterns that exist outside of the field component.
