This PR introduces a few changes to improve turbopack compatibility and
ensure e2e tests pass with turbopack enabled
## Changes to improve turbopack compatibility
- Use correct sideEffects configuration to fix scss issues
- Import scss directly instead of duplicating our scss rules
- Fix some scss rules that are not supported by turbopack
- Bump Next.js and all other dependencies used to build payload
## Changes to get tests to pass
For an unknown reason, flaky tests flake a lot more often in turbopack.
This PR does the following to get them to pass:
- add more `wait`s
- fix actual flakes by ensuring previous operations are properly awaited
## Blocking turbopack bugs
- [X] https://github.com/vercel/next.js/issues/76464
- Fix PR: https://github.com/vercel/next.js/pull/76545
- Once fixed: change `"sideEffectsDisabled":` back to `"sideEffects":`
## Non-blocking turbopack bugs
- [ ] https://github.com/vercel/next.js/issues/76956
## Related PRs
https://github.com/payloadcms/payload/pull/12653https://github.com/payloadcms/payload/pull/12652
I think it's easier to review this PR commit by commit, so I'll explain
it this way:
## Commits
1. [parallelize eslint script (still showing logs results in
serial)](c9ac49c12d):
Previously, `--concurrency 1` was added to the script to make the logs
more readable. However, turborepo has an option specifically for these
use cases: `--log-order=grouped` runs the tasks in parallel but outputs
them serially. As a result, the lint script is now significantly faster.
2. [run pnpm
lint:fix](9c128c276a)
The auto-fix was run, which resolved some eslint errors that were
slipped in due to the use of `no-verify`. Most of these were
`perfectionist` fixes (property ordering) and the removal of unnecessary
assertions. Starting with this PR, this won't happen again in the
future, as we'll be verifying the linter in every PR across the entire
codebase (see commit 7).
3. [fix eslint non-autofixable
errors](700f412a33)
All manual errors have been resolved except for the configuration errors
addressed in commit 5. Most were React compiler violations, which have
been disabled and commented out "TODO" for now. There's also an unused
`use no memo` and a couple of `require` errors.
4. [move react-compiler linter to eslint-config
package](4f7cb4d63a)
To simplify the eslint configuration. My concern was that there would be
a performance regression when used in non-react related packages, but
none was experienced. This is probably because it only runs on .tsx
files.
5. [remove redundant eslint config files and fix
allowDefaultProject](a94347995a)
The main feature introduced by `typescript-eslint` v8 was
`projectService`, which automatically searches each file for the closest
`tsconfig`, greatly simplifying configuration in monorepos
([source](https://typescript-eslint.io/blog/announcing-typescript-eslint-v8#project-service)).
Once I moved `projectService` to `packages/eslint-config`, all the other
configuration files could be easily removed.
I confirmed that pnpm lint still works on individual packages.
The other important change was that the pending eslint errors from
commits 2 and 3 were resolved. That is, some files were giving the
error: "[File] was not found by the project service. Consider either
including it in the tsconfig.json or including it in
allowDefaultProject." Below I copy the explanatory comment I left in the
code:
```ts
// This is necessary because `tsconfig.base.json` defines `"rootDir": "${configDir}/src"`,
// And the following files aren't in src because they aren't transpiled.
// This is typescript-eslint's way of adding files that aren't included in tsconfig.
// See: https://typescript-eslint.io/troubleshooting/typed-linting/#i-get-errors-telling-me--was-not-found-by-the-project-service-consider-either-including-it-in-the-tsconfigjson-or-including-it-in-allowdefaultproject
// The best practice is to have a tsconfig.json that covers ALL files and is used for
// typechecking (with noEmit), and a `tsconfig.build.json` that is used for the build
// (or alternatively, swc, tsup or tsdown). That's what we should ideally do, in which case
// this hardcoded list wouldn't be necessary. Note that these files don't currently go
// through ts, only through eslint.
```
6. [Differentiate errors from warnings in VScode ESLint
Rules](5914d2f48d)
There's no reason to do that. If an eslint rule isn't an error, it
should be disabled or converted to a warning.
7. [Disable skip lint, and lint over the entire repo now that it's
faster](e4b28f1360)
The GitHub action linted only the files that had changed in the PR.
While this seems like a good idea, once exceptions were introduced with
[skip lint], they opened the door to propagating more and more errors.
Often, the linter was skipped, not because someone introduced new
errors, but because they were trying to avoid those that had already
crept in, sometimes accidentally introducing new ones.
On the other hand, `pnpm lint` now runs in parallel (commit 1), so it's
not that slow. Additionally, it runs in parallel with other GitHub
actions like e2e tests, which take much longer, so it can't represent a
bottleneck in CI.
8. [fix lint in next
package](4506595f91)
Small fix missing from commit 5
9. [Merge remote-tracking branch 'origin/main' into
fix-eslint](563d4909c1)
10. [add again eslint.config.js in payload
package](78f6ffcae7)
The comment in the code explains it. Basically, after the merge from
main, the payload package runs out of memory when linting, probably
because it grew in recent PRs. That package will sooner or later
collapse for our tooling, so we may have to split it. It's already too
big.
## Future Actions
- Resolve React compiler violations, as mentioned in commit 3.
- Decouple the `tsconfig` used for typechecking and build across the
entire monorepo (as explained in point 5) to ensure ts coverage even for
files that aren't transpiled (such as scripts).
- Remove the few remaining `eslint.config.js`. I had to leave the
`richtext-lexical` and `next` ones for now. They could be moved to the
root config and scoped to their packages, as we do for example with
`templates/vercel-postgres/**`. However, I couldn't get it to work, I
don't know why.
- Make eslint in the test folder usable. Not only are we not linting
`test` in CI, but now the `pnpm eslint .` command is so large that my
computer freezes. If each suite were its own package, this would be
solved, and dynamic codegen + git hooks to modify tsconfig.base.json
wouldn't be necessary
([related](https://github.com/payloadcms/payload/pull/11984)).
When server rendering custom components within form state, those
components receive a path that is correct at render time, but
potentially stale after manipulating array and blocks rows. This causes
the field to briefly render incorrect values while the form state
request is in flight.
The reason for this is that paths are passed as a prop statically into
those components. Then when we manipulate rows, form state is modified,
potentially changing field paths. The component's `path` prop, however,
hasn't changed. This means it temporarily points to the wrong field in
form state, rendering the data of another row until the server responds
with a freshly rendered component.
This is not an issue with default Payload fields as they are rendered on
the client and can be passed dynamic props.
This is only an issue within custom server components, including rich
text fields which are treated as custom components. Since they are
rendered on the server and passed to the client, props are inaccessible
after render.
The fix for this is to provide paths dynamically through context. This
way as we make changes to form state, there is a mechanism in which
server components can receive the updated path without waiting on its
props to update.
**BREAKING CHANGE:**
This bumps the **minimum required Next.js** version from 15.0.0 to
15.2.3. This update is necessary due to a critical security
vulnerability found in earlier Next.js versions, which requires an
exception to our standard semantic versioning process.
Additionally, this bumps all templates to the latest Next.js and Payload
versions.
The form component's `initializing` and `processing` states do not
disable fields that are rendered outside of `DocumentFields`. Fields
currently rely on the `readOnly` prop provided by `DocumentFields` and
do not subscribe to these states for themselves. This means that fields
that are rendered outright, such as within the bulk edit drawer, they do
not receive a `readOnly` prop and are therefore never disabled.
The fix is add a `disabled` property to the `useField` hook. This
subscribes to the `initializing` and `processing` states in the same way
as `DocumentFields`, however, now each field can determine its own
disabled state instead of relying solely on the `readOnly` prop. Adding
this new prop has no overhead as `processing` and `initializing` is
already being subscribed to within `useField`.
### What?
Supersedes https://github.com/payloadcms/payload/pull/11490.
Refactors imports of `formatAdminURL` to import from `payload/shared`
instead of `@payloadcms/ui/shared`. The ui package now imports and
re-exports the function to prevent this from being a breaking change.
### Why?
This makes it easier for other packages/plugins to consume the
`formatAdminURL` function instead of needing to implement their own or
rely on the ui package for the utility.
This bumps next.js to 15.2.0 in our monorepo, as well as all @types/react and @types/react-dom versions. Additionally, it removes the obsolete `peerDependencies` property from our root package.json.
This PR also fixes 2 bugs introduced by Next.js 15.2.0. This highlights why running our test suite against the latest Next.js, to make sure Payload is compatible, version is important.
## 1. handleWhereChange running endlessly
Upgrading to Next.js 15.2.0 caused `handleWhereChange` to be continuously called by a `useEffect` when the list view filters were opened, leading to a React error - I did not investigate why upgrading the Next.js version caused that, but this PR fixes it by making use of the more predictable `useEffectEvent`.
## 2. Custom Block and Array label React key errors
Upgrading to Next.js 15.2.0 caused react key errors when rendering custom block and array row labels on the server. This has been fixed by rendering those with a key
## 3. Table React key errors
When rendering a `Table`, a React key error is thrown since Next.js 15.2.0
When rendering a list drawer, you can pass a custom `onSelect` callback
to execute when the user clicks on the linked cell within the table. The
underlying handler, however, only passes the `docID` and
`collectionSlug` args through the callback, rather than the document
itself. This makes it impossible to perform side-effects that require
the data of the row that was selected.
Instances of this callback were also largely untyped.
Needed for #11330.
The `localized` properly was not stripped out of referenced block fields, if any parent was localized. For normal fields, this is done in sanitizeConfig. As the same referenced block config can be used in both a localized and non-localized config, we are not able to strip it out inside sanitizeConfig by modifying the block config.
Instead, this PR had to bring back tedious logic to handle it everywhere the `field.localized` property is accessed. For backwards-compatibility, we need to keep the existing sanitizeConfig logic. In 4.0, we should remove it to benefit from better test coverage of runtime field.localized handling - for now, this is done for our test suite using the `PAYLOAD_DO_NOT_SANITIZE_LOCALIZED_PROPERTY` flag.
If you have multiple blocks that are used in multiple places, this can quickly blow up the size of your Payload Config. This will incur a performance hit, as more data is
1. sent to the client (=> bloated `ClientConfig` and large initial html) and
2. processed on the server (permissions are calculated every single time you navigate to a page - this iterates through all blocks you have defined, even if they're duplicative)
This can be optimized by defining your block **once** in your Payload Config, and just referencing the block slug whenever it's used, instead of passing the entire block config. To do this, the block can be defined in the `blocks` array of the Payload Config. The slug can then be passed to the `blockReferences` array in the Blocks Field - the `blocks` array has to be empty for compatibility reasons.
```ts
import { buildConfig } from 'payload'
import { lexicalEditor, BlocksFeature } from '@payloadcms/richtext-lexical'
// Payload Config
const config = buildConfig({
// Define the block once
blocks: [
{
slug: 'TextBlock',
fields: [
{
name: 'text',
type: 'text',
},
],
},
],
collections: [
{
slug: 'collection1',
fields: [
{
name: 'content',
type: 'blocks',
// Reference the block by slug
blockReferences: ['TextBlock'],
blocks: [], // Required to be empty, for compatibility reasons
},
],
},
{
slug: 'collection2',
fields: [
{
name: 'editor',
type: 'richText',
editor: lexicalEditor({
BlocksFeature({
// Same reference can be reused anywhere, even in the lexical editor, without incurred performance hit
blocks: ['TextBlock'],
})
})
},
],
},
],
})
```
## v4.0 Plans
In 4.0, we will remove the `blockReferences` property, and allow string block references to be passed directly to the blocks `property`. Essentially, we'd remove the `blocks` property and rename `blockReferences` to `blocks`.
The reason we opted to a new property in this PR is to avoid breaking changes. Allowing strings to be passed to the `blocks` property will prevent plugins that iterate through fields / blocks from compiling.
## PR Changes
- Testing: This PR introduces a plugin that automatically converts blocks to block references. This is done in the fields__blocks test suite, to run our existing test suite using block references.
- Block References support: Most changes are similar. Everywhere we iterate through blocks, we have to now do the following:
1. Check if `field.blockReferences` is provided. If so, only iterate through that.
2. Check if the block is an object (= actual block), or string
3. If it's a string, pull the actual block from the Payload Config or from `payload.blocks`.
The exception is config sanitization and block type generations. This PR optimizes them so that each block is only handled once, instead of every time the block is referenced.
## Benchmarks
60 Block fields, each block field having the same 600 Blocks.
### Before:
**Initial HTML:** 195 kB
**Generated types:** takes 11 minutes, 461,209 lines
https://github.com/user-attachments/assets/11d49a4e-5414-4579-8050-e6346e552f56
### After:
**Initial HTML:** 73.6 kB
**Generated types:** takes 2 seconds, 35,810 lines
https://github.com/user-attachments/assets/3eab1a99-6c29-489d-add5-698df67780a3
### After Permissions Optimization (follow-up PR)
Initial HTML: 73.6 kB
https://github.com/user-attachments/assets/a909202e-45a8-4bf6-9a38-8c85813f1312
## Future Plans
1. This PR does not yet deduplicate block references during permissions calculation. We'll optimize that in a separate PR, as this one is already large enough
2. The same optimization can be done to deduplicate fields. One common use-case would be link field groups that may be referenced in multiple entities, outside of blocks. We might explore adding a new `fieldReferences` property, that allows you to reference those same `config.blocks`.
Due to nature of server-side rendering, navigation within the admin
panel can lead to slow page response times. This can lead to the feeling
of an unresponsive app after clicking a link, for example, where the
page remains in a stale state while the server is processing. This is
especially noticeable on slow networks when navigating to data heavy or
process intensive pages.
To alleviate the bad UX that this causes, the user needs immediate
visual indication that _something_ is taking place. This PR renders a
progress bar in the admin panel which is immediately displayed when a
user clicks a link, and incrementally grows in size until the new route
has loaded in.
Inspired by https://github.com/vercel/react-transition-progress.
Old:
https://github.com/user-attachments/assets/1820dad1-3aea-417f-a61d-52244b12dc8d
New:
https://github.com/user-attachments/assets/99f4bb82-61d9-4a4c-9bdf-9e379bbafd31
To tie into the progress bar, you'll need to use Payload's new `Link`
component instead of the one provided by Next.js:
```diff
- import { Link } from 'next/link'
+ import { Link } from '@payloadcms/ui'
```
Here's an example:
```tsx
import { Link } from '@payloadcms/ui'
const MyComponent = () => {
return (
<Link href="/somewhere">
Go Somewhere
</Link>
)
}
```
In order to trigger route transitions for a direct router event such as
`router.push`, you'll need to wrap your function calls with the
`startRouteTransition` method provided by the `useRouteTransition` hook.
```ts
'use client'
import React, { useCallback } from 'react'
import { useTransition } from '@payloadcms/ui'
import { useRouter } from 'next/navigation'
const MyComponent: React.FC = () => {
const router = useRouter()
const { startRouteTransition } = useRouteTransition()
const redirectSomewhere = useCallback(() => {
startRouteTransition(() => router.push('/somewhere'))
}, [startRouteTransition, router])
// ...
}
```
In the future [Next.js might provide native support for
this](https://github.com/vercel/next.js/discussions/41934#discussioncomment-12077414),
and if it does, this implementation can likely be simplified.
Of course there are other ways of achieving this, such as with
[Suspense](https://react.dev/reference/react/Suspense), but they all
come with a different set of caveats. For example with Suspense, you
must provide a fallback component. This means that the user might be
able to immediately navigate to the new page, which is good, but they'd
be presented with a skeleton UI while the other parts of the page stream
in. Not necessarily an improvement to UX as there would be multiple
loading states with this approach.
There are other problems with using Suspense as well. Our default
template, for example, contains the app header and sidebar which are not
rendered within the root layout. This means that they need to stream in
every single time. On fast networks, this would also lead to a
noticeable "blink" unless there is some mechanism by which we can detect
and defer the fallback from ever rendering in such cases. Might still be
worth exploring in the future though.
Previously, we were unnecessarily passing the `ClientCollectionConfig`
down from the Table to the Client, even though the client cell
components could simply access it via the `useConfig` hook. This
resulted in redundant data being sent to the client for every single
table cell. Additionally, we were performing a deep copy of the
`ClientCollectionConfig`, which wasted both memory and CPU resources on
the server.
---------
Co-authored-by: Jacob Fletcher <jacobsfletch@gmail.com>
This PR modifies `tsconfig.base.json` by setting the following
strictness properties to true: `strict`, `noUncheckedIndexedAccess` and
`noImplicitOverride`.
In packages where compilation errors were observed, these settings were
opted out, and TODO comments were added to make it easier to track the
roadmap for converting everything to strict mode.
The following packages now have increased strictness, which prevents new
errors from being accidentally introduced:
- storage-vercel-blob
- storage-s3*
- storage-gcs
- plugin-sentry
- payload-cloud*
- email-resend*
- email-nodemailer*
*These packages already had `strict: true`, but now have
`noUncheckedIndexedAccess` and `noImplicitOverride`.
Note that this only affects the `/packages` folder, but not
`/templates`, `/test` or `/examples` which have a different `tsconfig`.
