This PR introduces a few changes to improve turbopack compatibility and
ensure e2e tests pass with turbopack enabled
## Changes to improve turbopack compatibility
- Use correct sideEffects configuration to fix scss issues
- Import scss directly instead of duplicating our scss rules
- Fix some scss rules that are not supported by turbopack
- Bump Next.js and all other dependencies used to build payload
## Changes to get tests to pass
For an unknown reason, flaky tests flake a lot more often in turbopack.
This PR does the following to get them to pass:
- add more `wait`s
- fix actual flakes by ensuring previous operations are properly awaited
## Blocking turbopack bugs
- [X] https://github.com/vercel/next.js/issues/76464
- Fix PR: https://github.com/vercel/next.js/pull/76545
- Once fixed: change `"sideEffectsDisabled":` back to `"sideEffects":`
## Non-blocking turbopack bugs
- [ ] https://github.com/vercel/next.js/issues/76956
## Related PRs
https://github.com/payloadcms/payload/pull/12653https://github.com/payloadcms/payload/pull/12652
I think it's easier to review this PR commit by commit, so I'll explain
it this way:
## Commits
1. [parallelize eslint script (still showing logs results in
serial)](c9ac49c12d):
Previously, `--concurrency 1` was added to the script to make the logs
more readable. However, turborepo has an option specifically for these
use cases: `--log-order=grouped` runs the tasks in parallel but outputs
them serially. As a result, the lint script is now significantly faster.
2. [run pnpm
lint:fix](9c128c276a)
The auto-fix was run, which resolved some eslint errors that were
slipped in due to the use of `no-verify`. Most of these were
`perfectionist` fixes (property ordering) and the removal of unnecessary
assertions. Starting with this PR, this won't happen again in the
future, as we'll be verifying the linter in every PR across the entire
codebase (see commit 7).
3. [fix eslint non-autofixable
errors](700f412a33)
All manual errors have been resolved except for the configuration errors
addressed in commit 5. Most were React compiler violations, which have
been disabled and commented out "TODO" for now. There's also an unused
`use no memo` and a couple of `require` errors.
4. [move react-compiler linter to eslint-config
package](4f7cb4d63a)
To simplify the eslint configuration. My concern was that there would be
a performance regression when used in non-react related packages, but
none was experienced. This is probably because it only runs on .tsx
files.
5. [remove redundant eslint config files and fix
allowDefaultProject](a94347995a)
The main feature introduced by `typescript-eslint` v8 was
`projectService`, which automatically searches each file for the closest
`tsconfig`, greatly simplifying configuration in monorepos
([source](https://typescript-eslint.io/blog/announcing-typescript-eslint-v8#project-service)).
Once I moved `projectService` to `packages/eslint-config`, all the other
configuration files could be easily removed.
I confirmed that pnpm lint still works on individual packages.
The other important change was that the pending eslint errors from
commits 2 and 3 were resolved. That is, some files were giving the
error: "[File] was not found by the project service. Consider either
including it in the tsconfig.json or including it in
allowDefaultProject." Below I copy the explanatory comment I left in the
code:
```ts
// This is necessary because `tsconfig.base.json` defines `"rootDir": "${configDir}/src"`,
// And the following files aren't in src because they aren't transpiled.
// This is typescript-eslint's way of adding files that aren't included in tsconfig.
// See: https://typescript-eslint.io/troubleshooting/typed-linting/#i-get-errors-telling-me--was-not-found-by-the-project-service-consider-either-including-it-in-the-tsconfigjson-or-including-it-in-allowdefaultproject
// The best practice is to have a tsconfig.json that covers ALL files and is used for
// typechecking (with noEmit), and a `tsconfig.build.json` that is used for the build
// (or alternatively, swc, tsup or tsdown). That's what we should ideally do, in which case
// this hardcoded list wouldn't be necessary. Note that these files don't currently go
// through ts, only through eslint.
```
6. [Differentiate errors from warnings in VScode ESLint
Rules](5914d2f48d)
There's no reason to do that. If an eslint rule isn't an error, it
should be disabled or converted to a warning.
7. [Disable skip lint, and lint over the entire repo now that it's
faster](e4b28f1360)
The GitHub action linted only the files that had changed in the PR.
While this seems like a good idea, once exceptions were introduced with
[skip lint], they opened the door to propagating more and more errors.
Often, the linter was skipped, not because someone introduced new
errors, but because they were trying to avoid those that had already
crept in, sometimes accidentally introducing new ones.
On the other hand, `pnpm lint` now runs in parallel (commit 1), so it's
not that slow. Additionally, it runs in parallel with other GitHub
actions like e2e tests, which take much longer, so it can't represent a
bottleneck in CI.
8. [fix lint in next
package](4506595f91)
Small fix missing from commit 5
9. [Merge remote-tracking branch 'origin/main' into
fix-eslint](563d4909c1)
10. [add again eslint.config.js in payload
package](78f6ffcae7)
The comment in the code explains it. Basically, after the merge from
main, the payload package runs out of memory when linting, probably
because it grew in recent PRs. That package will sooner or later
collapse for our tooling, so we may have to split it. It's already too
big.
## Future Actions
- Resolve React compiler violations, as mentioned in commit 3.
- Decouple the `tsconfig` used for typechecking and build across the
entire monorepo (as explained in point 5) to ensure ts coverage even for
files that aren't transpiled (such as scripts).
- Remove the few remaining `eslint.config.js`. I had to leave the
`richtext-lexical` and `next` ones for now. They could be moved to the
root config and scoped to their packages, as we do for example with
`templates/vercel-postgres/**`. However, I couldn't get it to work, I
don't know why.
- Make eslint in the test folder usable. Not only are we not linting
`test` in CI, but now the `pnpm eslint .` command is so large that my
computer freezes. If each suite were its own package, this would be
solved, and dynamic codegen + git hooks to modify tsconfig.base.json
wouldn't be necessary
([related](https://github.com/payloadcms/payload/pull/11984)).
## Fix
We were able to narrow it down to this call
816fb28f55/packages/plugin-cloud-storage/src/utilities/getFilePrefix.ts (L26-L41)
Adding `draft: true` fixes the issue. It seems that the `prefix` can
only be found in a draft, and without `draft: true` those drafts aren't
searched.
### Issue reproduction
In the community folder, enable versioning for the media collection and
install the `s3storage` plugin (see Git patch). I use `minio` to have a
local S3 compatible backend and then I run the app with:
`AWS_ACCESS_KEY_ID=minioadmin AWS_SECRET_ACCESS_KEY=minioadmin
START_MEMORY_DB=true pnpm dev _community`.
Next, open the media collection and create a new entry. Then open that
entry, remove the file it currently has, and upload a new file. Save as
draft.
Now the media can no longer be accessed and the thumbnails are broken.
If you make an edit but save it by publishing the issue goes away. I
also couldn't reproduce this by adding a text field, changing that, and
saving the document as draft.
```diff
diff --git test/_community/collections/Media/index.ts test/_community/collections/Media/index.ts
index bb5edd0349..689423053c 100644
--- test/_community/collections/Media/index.ts
+++ test/_community/collections/Media/index.ts
@@ -9,6 +9,9 @@ export const MediaCollection: CollectionConfig = {
read: () => true,
},
fields: [],
+ versions: {
+ drafts: true,
+ },
upload: {
crop: true,
focalPoint: true,
diff --git test/_community/config.ts test/_community/config.ts
index ee1aee6e46..c81ec5f933 100644
--- test/_community/config.ts
+++ test/_community/config.ts
@@ -7,6 +7,7 @@ import { devUser } from '../credentials.js'
import { MediaCollection } from './collections/Media/index.js'
import { PostsCollection, postsSlug } from './collections/Posts/index.js'
import { MenuGlobal } from './globals/Menu/index.js'
+import { s3Storage } from '@payloadcms/storage-s3'
const filename = fileURLToPath(import.meta.url)
const dirname = path.dirname(filename)
@@ -24,6 +25,21 @@ export default buildConfigWithDefaults({
// ...add more globals here
MenuGlobal,
],
+ plugins: [
+ s3Storage({
+ enabled: true,
+ bucket: 'amboss',
+ config: {
+ region: 'eu-west-1',
+ endpoint: 'http://localhost:9000',
+ },
+ collections: {
+ media: {
+ prefix: 'media',
+ },
+ },
+ }),
+ ],
onInit: async (payload) => {
await payload.create({
collection: 'users',
```
## Screen recording
https://github.com/user-attachments/assets/b13be4a3-e858-427a-8bfa-6592b87748ee
There are cases when a storage plugin is disabled during development and
enabled in production. This will result in import maps that differ
depending on if they're generated during development or production.
In a lot of cases, those import maps are generated during
development-only. During production, we just re-use what was generated
locally. This will cause missing import map entries for those plugins
that are disabled during development.
This PR ensures the import map entries are added regardless of the
enabled state of those plugins. This is necessary for our
generate-templates script to not omit the vercel blob storage import map
entry.
**BREAKING CHANGE:**
This bumps the **minimum required Next.js** version from 15.0.0 to
15.2.3. This update is necessary due to a critical security
vulnerability found in earlier Next.js versions, which requires an
exception to our standard semantic versioning process.
Additionally, this bumps all templates to the latest Next.js and Payload
versions.
Fixes
https://github.com/payloadcms/payload/issues/11731#issue-2923088087
Only 2 s3Storage() instances with `clientUploads: true` are currently
working. If you add a 3rd instance, uploading files errors with
"Collection credit-certificates was not found in S3 options".
There is already an implementation that changes the
`/storage-s3-generate-signed-url` URL for each new s3Storage plugin
instance so that multiple instances don't break each other. Currently
the code looks like this:
```ts
/**
* Tracks how many times the same handler was already applied.
* This allows to apply the same plugin multiple times, for example
* to use different buckets for different collections.
*/
let handlerCount = 0
for (const endpoint of config.endpoints) {
if (endpoint.path === serverHandlerPath) {
handlerCount++
}
}
if (handlerCount) {
serverHandlerPath = `${serverHandlerPath}-${handlerCount}`
}
```
When we print the endpoints generated by this code we get:
```ts
{
handler: [AsyncFunction (anonymous)],
method: 'post',
path: '/storage-s3-generate-signed-url'
},
{
handler: [AsyncFunction (anonymous)],
method: 'post',
path: '/storage-s3-generate-signed-url-1'
},
{
handler: [AsyncFunction (anonymous)],
method: 'post',
path: '/storage-s3-generate-signed-url-1'
}
```
As you can see, the path or the 3rd instance is the same as the 2nd
instance. Presumably this functionality was originally tested with 2
instances and not more, allowing this bug to slip through. This
completely breaks uploads for the 3rd instance.
We need to change the conditional that checks whether the plugin exists
already to use `.startsWith()`:
```ts
/**
* Tracks how many times the same handler was already applied.
* This allows to apply the same plugin multiple times, for example
* to use different buckets for different collections.
*/
let handlerCount = 0
for (const endpoint of config.endpoints) {
// We want to match on 'path', 'path-1', 'path-2', etc.
if (endpoint.path?.startsWith(serverHandlerPath)) {
handlerCount++
}
}
if (handlerCount) {
serverHandlerPath = `${serverHandlerPath}-${handlerCount}`
}
```
With the fix we can see that the endpoints increment correctly, allowing
for a true arbitrary number of s3Storage plugins with client uploads.
```ts
{
handler: [AsyncFunction (anonymous)],
method: 'post',
path: '/storage-s3-generate-signed-url'
},
{
handler: [AsyncFunction (anonymous)],
method: 'post',
path: '/storage-s3-generate-signed-url-1'
},
{
handler: [AsyncFunction (anonymous)],
method: 'post',
path: '/storage-s3-generate-signed-url-2'
}
```
Fixes https://github.com/payloadcms/payload/issues/11473
Previously, when `disablePayloadAccessControl: true` was defined, client
uploads were working improperly. The reason is that
`addDataAndFileToRequest` expects `staticHandler` to be defined and we
don't add in case if `disablePayloadAccessControl: true`.
This PR makes it so otherwise and if we have `clientUploads`, it pushes
the "proxied" handler that responses only when the file was requested in
the context of client upload (from `addDataAndFileToRequest`)
When uploading file via client side upload we invalidate it then on the
server side with re-uploading. This works fine with most adapters since
they just replace the old file under the same key. UploadThing works
differently and generates a new key every time.
Example of the issue:
<img width="611" alt="image"
src="https://github.com/user-attachments/assets/9c01b52a-d159-4f32-9f66-3b5fbadab7b4"
/>
Now, we clear the old file before doing re-upload.
### What?
Fixes client uploads when storage collection config has the `prefix`
property configured. Previously, it failed with "Object key was not
found".
### Why?
This is expected to work.
### How?
The client upload handler now receives to its props `prefix`. Then it
threads it to the server-side `staticHandler` through
`clientUploadContext` and then to `getFilePrefix`, which checks for
`clientUploadContext.prefix` and returns if there is.
Previously, `staticHandler` tried to load the file without including
prefix consideration.
This changes only these adapters:
* S3
* Azure
* GCS
With the Vercel Blob adapter, `prefix` works correctly.
This bumps next.js to 15.2.0 in our monorepo, as well as all @types/react and @types/react-dom versions. Additionally, it removes the obsolete `peerDependencies` property from our root package.json.
This PR also fixes 2 bugs introduced by Next.js 15.2.0. This highlights why running our test suite against the latest Next.js, to make sure Payload is compatible, version is important.
## 1. handleWhereChange running endlessly
Upgrading to Next.js 15.2.0 caused `handleWhereChange` to be continuously called by a `useEffect` when the list view filters were opened, leading to a React error - I did not investigate why upgrading the Next.js version caused that, but this PR fixes it by making use of the more predictable `useEffectEvent`.
## 2. Custom Block and Array label React key errors
Upgrading to Next.js 15.2.0 caused react key errors when rendering custom block and array row labels on the server. This has been fixed by rendering those with a key
## 3. Table React key errors
When rendering a `Table`, a React key error is thrown since Next.js 15.2.0
This PR modifies `tsconfig.base.json` by setting the following
strictness properties to true: `strict`, `noUncheckedIndexedAccess` and
`noImplicitOverride`.
In packages where compilation errors were observed, these settings were
opted out, and TODO comments were added to make it easier to track the
roadmap for converting everything to strict mode.
The following packages now have increased strictness, which prevents new
errors from being accidentally introduced:
- storage-vercel-blob
- storage-s3*
- storage-gcs
- plugin-sentry
- payload-cloud*
- email-resend*
- email-nodemailer*
*These packages already had `strict: true`, but now have
`noUncheckedIndexedAccess` and `noImplicitOverride`.
Note that this only affects the `/packages` folder, but not
`/templates`, `/test` or `/examples` which have a different `tsconfig`.
