In this case, the `blockType` property is created on the server, but -
prior to this fix - was discarded on the client in
[`fieldReducer.ts`](https://github.com/payloadcms/payload/blob/main/packages/ui/src/forms/Form/fieldReducer.ts#L186-L198)
via
[`mergerServerFormState.ts`](b9832f40e4/packages/ui/src/forms/Form/mergeServerFormState.ts (L29-L31)),
because the field's path neither existed in the client's form state, nor
was it marked as `addedByServer`.
This caused later calls to POST requests to form state to send without
the `blockType` key for block rows, which in turn caused
`addFieldStatePromise.ts` to throw the following error:
```
Block with type "undefined" was found in block data, but no block with that type is defined in the config for field with schema path ${schemaPath}.
```
This prevented the client side form state update from completing, and if
the form state was saved, broke the document.
This is a follow-up to #12131, which treated the symptom, but not the
cause. The original issue seems to have been introduced in
https://github.com/payloadcms/payload/releases/tag/v3.34.0. It's unclear
to me whether this issue is connected to block E2E tests having been
disabled in the same release in
https://github.com/payloadcms/payload/pull/11988.
## How to reproduce
### Collection configuration
```ts
const RICH_TEXT_BLOCK_TYPE = 'richTextBlockType'
const RichTextBlock: Block = {
slug: RICH_TEXT_BLOCK_TYPE,
interfaceName: 'RichTextBlock',
fields: [
{
name: 'richTextBlockField',
label: 'Rich Text Field in Block Field',
type: 'richText',
editor: lexicalEditor({}),
required: true,
},
],
}
const MyCollection: CollectionConfig = {
slug: 'my-collection-slug,
fields: [
{
name: 'arrayField',
label: 'Array Field',
type: 'array',
fields: [
{
name: 'blockField',
type: 'blocks',
blocks: [RichTextBlock],
required: true,
},
],
},
]
}
export default MyCollection
```
### Steps
- Press "Add Array Field"
--> ✅ 1st block with rich text is added
- Press "Add Array Field" a 2nd time
### Result
- 🛑 2nd block is indefinitely in loading state (side-note: the form UI
should preferably explicitly indicate the error).
- 🛑 If saving the document, it is corrupted and will only show a blank
page (also not indicating any error).
Client side:
<img width="1268" alt="Untitled"
src="https://github.com/user-attachments/assets/4b32fdeb-af76-41e2-9181-d2dbd686618a"
/>
API error:
<img width="1272" alt="image"
src="https://github.com/user-attachments/assets/35dc65f7-88ac-4397-b8d4-353bcf6a4bfd"
/>
Client side, when saving and re-opening document (API error of `GET
/admin/collections/${myCollection}/${documentId}` is the same (arguably
the HTTP response status code shouldn't be `200`)):
<img width="1281" alt="image"
src="https://github.com/user-attachments/assets/2e916eb5-6f10-4e82-9b84-1dc41db21d47"
/>
### Result after fix
- `blockType` is sent from the client to the server.
- ✅ 2nd block with rich text is added.
- ✅ Document does not break when saving & re-opening.
<img width="1277" alt="Untitled"
src="https://github.com/user-attachments/assets/84d0c88b-64b2-48c4-864d-610d524ac8fc"
/>
---------
Co-authored-by: Jacob Fletcher <jacobsfletch@gmail.com>
Fixes form state race conditions. Modifying state while a request is in
flight or while the response is being processed could result in those
changes being overridden.
This was happening for a few reasons:
1. Our merge logic was incorrect. We were disregarding local changes to
state that may have occurred while form state requests are pending. This
was because we were iterating over local state, then while building up
new state, we were ignoring any fields that did not exist in the server
response, like this:
```ts
for (const [path, newFieldState] of Object.entries(existingState)) {
if (!incomingState[path]) {
continue
}
// ...
}
```
To fix this, we need to use local state as the source of truth. Then
when the server state arrives, we need to iterate over _that_. If a
field matches in local state, merge in any new properties. This will
ensure all changes to the underlying state are preserved, including any
potential addition or deletions.
However, this logic breaks down if the server might have created _new_
fields, like when populating array rows. This means they, too, would be
ignored. To get around this, there is a new `addedByServer` property
that flags new fields to ensure they are kept.
This new merge strategy also saves an additional loop over form state.
1. We were merging form state based on a mutable ref. This meant that
changes made within one action cause concurrent actions to have dirty
reads. The fix for this is to merge in an isolated manner by copying
state. This will remove any object references. It is generally not good
practice to mutate state without setting it, anyways, as this causes
mismatches between what is rendered and what is in memory.
1. We were merging server form state directly within an effect, then
replacing state entirely. This meant that if another action took place
at the exact moment in time _after_ merge but _before_ dispatch, the
results of that other action would be completely overridden. The fix for
this is to perform the merge within the reducer itself. This will ensure
that we are working with a trustworthy snapshot of state at the exact
moment in time that the action was invoked, and that React can properly
queue the event within its lifecycle.
Significantly optimizes the component rendering strategy within the form
state endpoint by precisely rendering only the fields that require it.
This cuts down on server processing and network response sizes when
invoking form state requests **that manipulate array and block rows
which contain server components**, such as rich text fields, custom row
labels, etc. (results listed below).
Here's a breakdown of the issue:
Previously, when manipulating array and block fields, _all_ rows would
render any server components that might exist within them, including
rich text fields. This means that subsequent changes to these fields
would potentially _re-render_ those same components even if they don't
require it.
For example, if you have an array field with a rich text field within
it, adding the first row would cause the rich text field to render,
which is expected. However, when you add a second row, the rich text
field within the first row would render again unnecessarily along with
the new row.
This is especially noticeable for fields with many rows, where every
single row processes its server components and returns RSC data. And
this does not only affect nested rich text fields, but any custom
component defined on the field level, as these are handled in the same
way.
The reason this was necessary in the first place was to ensure that the
server components receive the proper data when they are rendered, such
as the row index and the row's data. Changing one of these rows could
cause the server component to receive the wrong data if it was not
freshly rendered.
While this is still a requirement that rows receive up-to-date props, it
is no longer necessary to render everything.
Here's a breakdown of the actual fix:
This change ensures that only the fields that are actually being
manipulated will be rendered, rather than all rows. The existing rows
will remain in memory on the client, while the newly rendered components
will return from the server. For example, if you add a new row to an
array field, only the new row will render its server components.
To do this, we send the path of the field that is being manipulated to
the server. The server can then use this path to determine for itself
which fields have already been rendered and which ones need required
rendering.
## Results
The following results were gathered by booting up the `form-state` test
suite and seeding 100 array rows, each containing a rich text field. To
invoke a form state request, we navigate to a document within the
"posts" collection, then add a new array row to the list. The result is
then saved to the file system for comparison.
| Test Suite | Collection | Number of Rows | Before | After | Percentage
Change |
|------|------|---------|--------|--------|--------|
| `form-state` | `posts` | 101 | 1.9MB / 266ms | 80KB / 70ms | ~96%
smaller / ~75% faster |
---------
Co-authored-by: James <james@trbl.design>
Co-authored-by: Alessio Gravili <alessio@gravili.de>
Implements a select-like API into the form state endpoint. This follows
the same spec as the Select API on existing Payload operations, but
works on form state rather than at the db level. This means you can send
the `select` argument through the form state handler, and it will only
process and return the fields you've explicitly identified.
This is especially useful when you only need to generate a partial form
state, for example within the bulk edit form where you select only a
subset of fields to edit. There is no need to iterate all fields of the
schema, generate default values for each, and return them all through
the network. This will also simplify and reduce the amount of
client-side processing required, where we longer need to strip
unselected fields before submission.
Implements a form state task queue. This will prevent onChange handlers
within the form component from processing unnecessarily often, sometimes
long after the user has stopped making changes. This leads to a
potentially huge number of network requests if those changes were made
slower than the debounce rate. This is especially noticeable on slow
networks.
Does so through a new `useQueue` hook. This hook maintains a stack of
events that need processing but only processes the final event to
arrive. Every time a new event is pushed to the stack, the currently
running process is aborted (if any), and that event becomes the next in
the queue. This results in a shocking reduction in the time it takes
between final change to form state and the final network response, from
~1.5 minutes to ~3 seconds (depending on the scenario, see below).
This likely fixes a number of existing open issues. I will link those
issues here once they are identified and verifiably fixed.
Before:
I'm typing slowly here to ensure my changes aren't debounce by the form.
There are a total of 60 characters typed, triggering 58 network requests
and taking around 1.5 minutes to complete after the final change was
made.
https://github.com/user-attachments/assets/49ba0790-a8f8-4390-8421-87453ff8b650
After:
Here there are a total of 69 characters typed, triggering 11 network
requests and taking only about 3 seconds to complete after the final
change was made.
https://github.com/user-attachments/assets/447f8303-0957-41bd-bb2d-9e1151ed9ec3
