Migrates the `db-mongodb` package to use `strict: true` and
`noUncheckedIndexedAccess: true` TSConfig properties.
This greatly improves code quality and prevents some runtime errors or
gives better error messages.
This feature allows you to specify `collection` for the join field as
array.
This can be useful for example to describe relationship linking like
this:
```ts
{
slug: 'folders',
fields: [
{
type: 'join',
on: 'folder',
collection: ['files', 'documents', 'folders'],
name: 'children',
},
{
type: 'relationship',
relationTo: 'folders',
name: 'folder',
},
],
},
{
slug: 'files',
upload: true,
fields: [
{
type: 'relationship',
relationTo: 'folders',
name: 'folder',
},
],
},
{
slug: 'documents',
fields: [
{
type: 'relationship',
relationTo: 'folders',
name: 'folder',
},
],
},
```
Documents and files can be placed to folders and folders themselves can
be nested to other folders (root folders just have `folder` as `null`).
Output type of `Folder`:
```ts
export interface Folder {
id: string;
children?: {
docs?:
| (
| {
relationTo?: 'files';
value: string | File;
}
| {
relationTo?: 'documents';
value: string | Document;
}
| {
relationTo?: 'folders';
value: string | Folder;
}
)[]
| null;
hasNextPage?: boolean | null;
} | null;
folder?: (string | null) | Folder;
updatedAt: string;
createdAt: string;
}
```
While you could instead have many join fields (for example
`childrenFolders`, `childrenFiles`) etc - this doesn't allow you to
sort/filter and paginate things across many collections, which isn't
trivial. With SQL we use `UNION ALL` query to achieve that.
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
The `localized` properly was not stripped out of referenced block fields, if any parent was localized. For normal fields, this is done in sanitizeConfig. As the same referenced block config can be used in both a localized and non-localized config, we are not able to strip it out inside sanitizeConfig by modifying the block config.
Instead, this PR had to bring back tedious logic to handle it everywhere the `field.localized` property is accessed. For backwards-compatibility, we need to keep the existing sanitizeConfig logic. In 4.0, we should remove it to benefit from better test coverage of runtime field.localized handling - for now, this is done for our test suite using the `PAYLOAD_DO_NOT_SANITIZE_LOCALIZED_PROPERTY` flag.
If you have multiple blocks that are used in multiple places, this can quickly blow up the size of your Payload Config. This will incur a performance hit, as more data is
1. sent to the client (=> bloated `ClientConfig` and large initial html) and
2. processed on the server (permissions are calculated every single time you navigate to a page - this iterates through all blocks you have defined, even if they're duplicative)
This can be optimized by defining your block **once** in your Payload Config, and just referencing the block slug whenever it's used, instead of passing the entire block config. To do this, the block can be defined in the `blocks` array of the Payload Config. The slug can then be passed to the `blockReferences` array in the Blocks Field - the `blocks` array has to be empty for compatibility reasons.
```ts
import { buildConfig } from 'payload'
import { lexicalEditor, BlocksFeature } from '@payloadcms/richtext-lexical'
// Payload Config
const config = buildConfig({
// Define the block once
blocks: [
{
slug: 'TextBlock',
fields: [
{
name: 'text',
type: 'text',
},
],
},
],
collections: [
{
slug: 'collection1',
fields: [
{
name: 'content',
type: 'blocks',
// Reference the block by slug
blockReferences: ['TextBlock'],
blocks: [], // Required to be empty, for compatibility reasons
},
],
},
{
slug: 'collection2',
fields: [
{
name: 'editor',
type: 'richText',
editor: lexicalEditor({
BlocksFeature({
// Same reference can be reused anywhere, even in the lexical editor, without incurred performance hit
blocks: ['TextBlock'],
})
})
},
],
},
],
})
```
## v4.0 Plans
In 4.0, we will remove the `blockReferences` property, and allow string block references to be passed directly to the blocks `property`. Essentially, we'd remove the `blocks` property and rename `blockReferences` to `blocks`.
The reason we opted to a new property in this PR is to avoid breaking changes. Allowing strings to be passed to the `blocks` property will prevent plugins that iterate through fields / blocks from compiling.
## PR Changes
- Testing: This PR introduces a plugin that automatically converts blocks to block references. This is done in the fields__blocks test suite, to run our existing test suite using block references.
- Block References support: Most changes are similar. Everywhere we iterate through blocks, we have to now do the following:
1. Check if `field.blockReferences` is provided. If so, only iterate through that.
2. Check if the block is an object (= actual block), or string
3. If it's a string, pull the actual block from the Payload Config or from `payload.blocks`.
The exception is config sanitization and block type generations. This PR optimizes them so that each block is only handled once, instead of every time the block is referenced.
## Benchmarks
60 Block fields, each block field having the same 600 Blocks.
### Before:
**Initial HTML:** 195 kB
**Generated types:** takes 11 minutes, 461,209 lines
https://github.com/user-attachments/assets/11d49a4e-5414-4579-8050-e6346e552f56
### After:
**Initial HTML:** 73.6 kB
**Generated types:** takes 2 seconds, 35,810 lines
https://github.com/user-attachments/assets/3eab1a99-6c29-489d-add5-698df67780a3
### After Permissions Optimization (follow-up PR)
Initial HTML: 73.6 kB
https://github.com/user-attachments/assets/a909202e-45a8-4bf6-9a38-8c85813f1312
## Future Plans
1. This PR does not yet deduplicate block references during permissions calculation. We'll optimize that in a separate PR, as this one is already large enough
2. The same optimization can be done to deduplicate fields. One common use-case would be link field groups that may be referenced in multiple entities, outside of blocks. We might explore adding a new `fieldReferences` property, that allows you to reference those same `config.blocks`.
Previously, queries like this didn't work:
```ts
const res = await payload.find({
collection: 'polymorphic-relationships',
where: {
polymorphicLocalized: {
equals: {
relationTo: 'movies',
value: movie.id,
},
},
},
})
```
This was due to the incorrectly passed path to MongoDB without
`.{locale}` suffix.
Additionally, to MongoDB now we send:
```
{
$or: [
{
polymorphic: {
$eq: {
relationTo: formattedValue.relationTo,
value: formattedValue.value,
},
},
},
{
polymorphic: {
$eq: {
relationTo: 'movies',
value: 'some-id',
},
},
},
],
},
```
Instead of:
```
{
$and: [
{
'polymorphic.relationTo': {
$eq: 'movies ',
},
},
{
'polymorphic.value': {
$eq: 'some-id ',
},
},
],
}
```
To match the _exact_ value. This is essential when we do querying by
relationships with `hasMany: true` and custom IDs that can be repeated.
`$or` is needed if for some reason keys are stored in the DB in a
different order
### What?
Upgrades mongoose from 6 to latest `v8.8.1`
Fixes https://github.com/payloadcms/payload/issues/9171
### Why?
Compatibilty with Mongodb Atlas
### How?
- Updates deps
- Changed ObjectId from bson-objectid to use `new Type.ObjectId` from
mongoose for compatibility (only inside of db-mongodb)
- Internal type adjustments
https://github.com/payloadcms/payload/discussions/9088
BREAKING CHANGES:
All projects with existing data having versions enabled, or relationship or upload fields will want to create the predefined migration that converts all strings to ObjectIDs where needed. This can be created using `payload migrate:create --file @payloadcms/mongodb/relationships-v2-v3`.
For projects making use of the exposed Models from mongoose, review the
upgrade guides from [v6 to
v7](https://mongoosejs.com/docs/7.x/docs/migrating_to_7.html) and [v7 to
v8](https://mongoosejs.com/docs/migrating_to_8.html) and make
adjustments as needed.
---------
Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
### What?
Fixes the issue with `in` querying when the collection has a join field.
### Why?
When using `.aggregate`, MongoDB doesn't cast a comma delimited value
for the `$in` operator to an array automatically as it's not handled by
Mongoose.
### How?
Sanitizes the incoming value to an array if it should.
Fixes https://github.com/payloadcms/payload/issues/8901
## Description
- Adds a new "join" field type to Payload and is supported by all database adapters
- The UI uses a table view for the new field
- `db-mongodb` changes relationships to be stored as ObjectIDs instead of strings (for now querying works using both types internally to the DB so no data migration should be necessary unless you're querying directly, see breaking changes for details
- Adds a reusable traverseFields utility to Payload to make it easier to work with nested fields, used internally and for plugin maintainers
```ts
export const Categories: CollectionConfig = {
slug: 'categories',
fields: [
{
name: 'relatedPosts',
type: 'join',
collection: 'posts',
on: 'category',
}
]
}
```
BREAKING CHANGES:
All mongodb relationship and upload values will be stored as MongoDB ObjectIDs instead of strings going forward. If you have existing data and you are querying data directly, outside of Payload's APIs, you get different results. For example, a `contains` query will no longer works given a partial ID of a relationship since the ObjectID requires the whole identifier to work.
---------
Co-authored-by: Jacob Fletcher <jacobsfletch@gmail.com>
Co-authored-by: James <james@trbl.design>
# Breaking Changes
### New file import locations
Exports from the `payload` package have been _significantly_ cleaned up.
Now, just about everything is able to be imported from `payload`
directly, rather than an assortment of subpath exports. This means that
things like `import { buildConfig } from 'payload/config'` are now just
imported via `import { buildConfig } from 'payload'`. The mental model
is significantly simpler for developers, but you might need to update
some of your imports.
Payload now exposes only three exports:
1. `payload` - all types and server-only Payload code
2. `payload/shared` - utilities that can be used in either the browser
or in Node environments
3. `payload/node` - heavy utilities that should only be imported in Node
scripts and never be imported into bundled code like Next.js
### UI library pre-bundling
With this release, we've dramatically sped up the compile time for
Payload by pre-bundling our entire UI package for use inside of the
Payload admin itself. There are new exports that should be used within
Payload custom components:
1. `@payloadcms/ui/client` - all client components
2. `@payloadcms/ui/server` - all server components
For all of your custom Payload admin UI components, you should be
importing from one of these two pre-compiled barrel files rather than
importing from the more deeply nested exports directly. That will keep
compile times nice and speedy, and will also make sure that the bundled
JS for your admin UI is kept small.
For example, whereas before, if you imported the Payload `Button`, you
would have imported it like this:
```ts
import { Button } from '@payloadcms/ui/elements/Button'
```
Now, you would import it like this:
```ts
import { Button } from '@payloadcms/ui/client'
```
This is a significant DX / performance optimization that we're pretty
pumped about.
However, if you are importing or re-using Payload UI components
_outside_ of the Payload admin UI, for example in your own frontend
apps, you can import from the individual component exports which will
make sure that the bundled JS is kept to a minimum in your frontend
apps. So in your own frontend, you can continue to import directly to
the components that you want to consume rather than importing from the
pre-compiled barrel files.
Individual component exports will now come with their corresponding CSS
and everything will work perfectly as-expected.
### Specific exports have changed
- `'@payloadcms/ui/templates/Default'` and
`'@payloadcms/ui/templates/Minimal`' are now exported from
`'@payloadcms/next/templates'`
- Old: `import { LogOut } from '@payloadcms/ui/icons/LogOut'` new:
`import { LogOutIcon } from '@payloadcms/ui/icons/LogOut'`
## Background info
In effort to make local dev as fast as possible, we need to import as
few files as possible so that the compiler has less to process. One way
we've achieved this in the Admin Panel was to _remove_ all .scss imports
from all components in the `@payloadcms/ui` module using a build
process. This stripped all `import './index.scss'` statements out of
each component before injecting them into `dist`. Instead, it bundles
all of the CSS into a single `main.css` file, and we import _that_ at
the root of the app.
While this concept is _still_ the right solution to the problem, this
particular approach is not viable when using these components outside
the Admin Panel, where not only does this root stylesheet not exist, but
where it would also bloat your app with unused styles. Instead, we need
to _keep_ these .scss imports in place so they are imported directly
alongside your components, as expected. Then, we need create a _new_
build step that _separately_ compiles the components _without_ their
stylesheets—this way your app can consume either as needed from the new
`client` and `server` barrel files within `@payloadcms/ui`, i.e. from
within `@payloadcms/next` and all other admin-specific packages and
plugins.
This way, all other applications will simply import using the direct
file paths, just as they did before. Except now they come with
stylesheets.
And we've gotten a pretty awesome initial compilation performance boost.
---------
Co-authored-by: James <james@trbl.design>
Co-authored-by: Alessio Gravili <alessio@gravili.de>
* chore: cleanup tsconfigs and fix db-mongodb builds
* chore: make the db adapters depend on payload
* chore: fix tsconfig for test directory
* chore: fix packages/db-mongodb not building
