Commit Graph

72 Commits

Author SHA1 Message Date
Sasha
24ace70b58 fix(db-mongodb): support 2x and more relationship sorting (#13819)
Previously, sorting like:
`sort: 'relationship.anotherRelationship.title'` (and more nested)
didn't work with the MongoDB adapter, this PR fixes this.
2025-09-16 21:03:48 +03:00
Sasha
75385de01f fix: filtering by polymorphic relationships inside other fields (#13265)
Previously, filtering by a polymorphic relationship inside an array /
group (unless the `name` is `version`) / tab caused `QueryError: The
following path cannot be queried:`.
2025-07-25 09:10:21 -04:00
Elliott W
41cff6d436 fix(db-mongodb): improve compatibility with Firestore database (#12763)
### What?

Adds four more arguments to the `mongooseAdapter`:

```typescript
  useJoinAggregations?: boolean  /* The big one */
  useAlternativeDropDatabase?: boolean
  useBigIntForNumberIDs?: boolean
  usePipelineInSortLookup?: boolean
```

Also export a new `compatabilityOptions` object from
`@payloadcms/db-mongodb` where each key is a mongo-compatible database
and the value is the recommended `mongooseAdapter` settings for
compatability.

### Why?

When using firestore and visiting
`/admin/collections/media/payload-folders`, we get:

```
MongoServerError: invalid field(s) in lookup: [let, pipeline], only lookup(from, localField, foreignField, as) is supported
```

Firestore doesn't support the full MongoDB aggregation API used by
Payload which gets used when building aggregations for populating join
fields.

There are several other compatability issues with Firestore:
- The invalid `pipeline` property is used in the `$lookup` aggregation
in `buildSortParams`
- Firestore only supports number IDs of type `Long`, but Mongoose
converts custom ID fields of type number to `Double`
- Firestore does not support the `dropDatabase` command
- Firestore does not support the `createIndex` command (not addressed in
this PR)

### How?

 ```typescript
useJoinAggregations?: boolean  /* The big one */
```
When this is `false` we skip the `buildJoinAggregation()` pipeline and resolve the join fields through multiple queries. This can potentially be used with AWS DocumentDB and Azure Cosmos DB to support join fields, but I have not tested with either of these databases.

 ```typescript
useAlternativeDropDatabase?: boolean
```
When `true`, monkey-patch (replace) the `dropDatabase` function so that
it calls `collection.deleteMany({})` on every collection instead of
sending a single `dropDatabase` command to the database

 ```typescript
useBigIntForNumberIDs?: boolean
```
When `true`, use `mongoose.Schema.Types.BigInt` for custom ID fields of type `number` which converts to a firestore `Long` behind the scenes

```typescript
  usePipelineInSortLookup?: boolean
```
When `false`, modify the sortAggregation pipeline in `buildSortParams()` so that we don't use the `pipeline` property in the `$lookup` aggregation. Results in slightly worse performance when sorting by relationship properties.

### Limitations

This PR does not add support for transactions or creating indexes in firestore.

### Fixes

Fixed a bug (and added a test) where you weren't able to sort by multiple properties on a relationship field.

### Future work

1. Firestore supports simple `$lookup` aggregations but other databases might not. Could add a `useSortAggregations` property which can be used to disable aggregations in sorting.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
2025-07-16 15:17:43 -04:00
Sasha
0e8ac0bad5 fix(db-postgres): joins with hasMany: true relationships nested to an array (#12980)
Fixes https://github.com/payloadcms/payload/issues/12679
2025-06-30 21:25:29 +03:00
Sasha
b74969d720 fix(db-postgres): querying on hasMany: true select field in a relationship (#12916)
Fixes https://github.com/payloadcms/payload/issues/11635
2025-06-24 21:25:48 +03:00
Sasha
245a2dee7e fix(db-mongodb): 4x and more level deep relationships querying (#12800)
Fixes https://github.com/payloadcms/payload/issues/12721
2025-06-13 14:11:13 +00:00
Alessio Gravili
7c05c775cb docs: improve jobs autorun docs, adds e2e test (#12196)
This clarifies that jobs.autoRun only *runs* already-queued jobs. It does not queue the jobs for you.

Also adds an e2e test as this functionality had no e2e coverage
2025-06-05 09:19:19 -07:00
Sasha
fd67d461ac fix(db-mongodb): sort by fields in relationships with draft: true (#12387)
Fixes sorting by fields in relationships, e.g `sort: "author.name"` when
using `draft: true`. The existing test that includes check with `draft:
true` was accidentally passing because it used to sort by the
relationship field itself.
2025-05-12 22:35:16 +03:00
Sasha
857e984fbb fix(db-mongodb): querying relationships with where clause as an object with several conditions (#11953)
Fixes https://github.com/payloadcms/payload/issues/11927

When trying to use the following notation:
```ts
const { docs } = await payload.find({
  collection: 'movies',
  depth: 0,
  where: {
    'director.name': { equals: 'Director1' },
    'director.localized': { equals: 'Director1_Localized' },
  },
})
```
Currently, it respects only the latest condition and the first is
ignored.

However, this works fine:
```ts
const { docs } = await payload.find({
  collection: 'movies',
  depth: 0,
  where: {
    and: [
      {
        'director.name': { equals: 'Director1' },
      },
      {
        'director.localized': { equals: 'Director1_Localized' },
      },
    ],
  },
})
```

But this should be an equivalent to
```
 where: {
    'director.name': { equals: 'Director1' },
    'director.localized': { equals: 'Director1_Localized' },
  },
```
2025-04-03 09:07:10 -04:00
Sasha
93cc66d745 test: rearrange relationships test blocks properly (#11858)
Previously, many test cases in `int/relationships` were wrapped to the
"custom IDs" describe block even though they aren't related to custom
IDs at all. This rearranges them as they should be.
2025-03-25 15:35:33 +00:00
Sasha
4081953c18 feat(db-mongodb): support sorting by fields in other collections through a relationship field (#11803)
This is already supported in Postgres / SQLite.
For example:
```
const result = await payload.find({
  collection: 'directors',
  depth: 0,
  sort: '-movies.name', // movies is a relationship field here
})
```

Removes the condition in tests:
```
 // no support for sort by relation in mongodb
 if (isMongoose(payload)) {
  return
}
```
2025-03-20 20:49:21 +00:00
Sasha
88548fcbe6 fix(db-postgres): querying other collections via relationships inside blocks (#11255)
### What?
Previously, in postgres query like:
```ts
const result = await payload.find({
  collection: 'blocks',
  where: { 'blocks.director.name': { equals: 'Test Director' } },
})
```
where `blocks` is a blocks field, `director` is a relationship field and
`name` is a text field inside `directors`, failed with:

![image](https://github.com/user-attachments/assets/f4b62b69-bd17-4ef0-9f0c-08057e9f2d57)

### Why?
The generated query before was a bit wrong.
Before:
```sql
select distinct
  "blocks"."id",
  "blocks"."created_at",
  "blocks"."created_at"
from
  "blocks"
  left join "directors" "a5ad426a_eda4_4067_af7e_5b294d7f0968" on "a5ad426a_eda4_4067_af7e_5b294d7f0968"."id" = "blocks_blocks_some"."director_id"
   left join "blocks_blocks_some" on "blocks"."id" = "blocks_blocks_some"."_parent_id"
where
  "a5ad426a_eda4_4067_af7e_5b294d7f0968"."name" = 'Test Director'
order by
  "blocks"."created_at" desc
limit
	10
```
Notice `left join directors` _before_ join of `blocks_blocks_some`.
`blocks_blocks_some` doesn't exist yet, this PR changes so now we
generate
```sql
select distinct
  "blocks"."id",
  "blocks"."created_at",
  "blocks"."created_at"
from
  "blocks"
  left join "blocks_blocks_some" on "blocks"."id" = "blocks_blocks_some"."_parent_id"
  left join "directors" "a5ad426a_eda4_4067_af7e_5b294d7f0968" on "a5ad426a_eda4_4067_af7e_5b294d7f0968"."id" = "blocks_blocks_some"."director_id"
where
  "a5ad426a_eda4_4067_af7e_5b294d7f0968"."name" = 'Test Director'
order by
  "blocks"."created_at" desc
limit
	10
```
2025-02-18 14:51:17 -05:00
Sasha
513ba636af fix(db-postgres): ensure countDistinct works correctly and achieve better performance when the query has table joins (#11208)
The fix, added in https://github.com/payloadcms/payload/pull/11096
wasn't sufficient enough. It did handle the case when the same query
path / table was joined twice and caused incorrect `totalDocs`, but it
didn't handle the case when `JOIN` returns more than 1 rows, which 2
added new assertions here check.

Now, we use `COUNT(*)` only if we don't have any joined tables. If we
do, instead of using `SELECT (COUNT DISTINCT id)` which as described in
the previous PR is _very slow_ for large tables, we use the following
query:

```sql
SELECT COUNT(1) OVER() as count -- window function, executes for each row only once
FROM users
LEFT JOIN -- ... here additional rows are added
WHERE -- ...
GROUP BY users.id -- this ensures we're counting only users without additional rows from joins. 
LIMIT 1 -- Since COUNT(1) OVER() executes and resolves before doing LIMIT, we can safely apply LIMIT 1.
```
2025-02-16 14:08:08 +02:00
Sasha
98fec35368 fix(db-postgres): incorrect pagination results when querying hasMany relationships multiple times (#11096)
Fixes https://github.com/payloadcms/payload/issues/10810

This was caused by using `COUNT(*)` aggregation instead of
`COUNT(DISTINCT table.id)`. However, we want to use `COUNT(*)` because
`COUNT(DISTINCT table.id)` is slow on large tables. Now we fallback to
`COUNT(DISTINCT table.id)` only when `COUNT(*)` cannot work properly.

Example of a query that leads to incorrect `totalDocs`:
```ts
const res = await payload.find({
  collection: 'directors',
  limit: 10,
  where: {
    or: [
      {
        movies: {
          equals: movie2.id,
        },
      },
      {
        movies: {
          equals: movie1.id,
        },
      },
      {
        movies: {
          equals: movie1.id,
        },
      },
    ],
  },
})
```
2025-02-11 01:16:18 +02:00
Sasha
e1dcb9594c fix(db-postgres): write operations on polymorphic joined collections throw error (#10854)
Fixes https://github.com/payloadcms/payload/issues/10845
See
https://github.com/payloadcms/payload/issues/10845#issuecomment-2620201486
2025-01-31 18:25:47 +02:00
Sasha
d601300034 fix(db-mongodb): querying polymorphic relationships with the all operator (#10704)
Fixes https://github.com/payloadcms/payload/issues/10678
2025-01-23 18:23:50 +02:00
Patrik
ad553e967b fix: updates field validation error messages to use labels if applicable (#10601)
### What?

Previously, field error messages displayed in toast notifications used
the field path to reference fields that failed validation. This
path-based approach was necessary to distinguish between fields that
might share the same name when nested inside arrays, groups, rows, or
collapsible fields.

However, the human readability of these paths was lacking, especially
for unnamed fields like rows and collapsible fields. For example:

- A text field inside a row could display as: `_index-0.text`
- A text field nested within multiple arrays could display as:
`items.0.subArray.0.text`

These outputs are technically correct but not user-friendly.

### Why?

While the previous format was helpful for pinpointing the specific field
that caused the validation error, it could be more user-friendly and
clearer to read. The goal is to maintain the same level of accuracy
while improving the readability for both developers and content editors.

### How?

To improve readability, the following changes were made:

1. Use Field Labels Instead of Field Paths:
- The ValidationError component now uses the label prop from the field
config (if available) instead of the field’s name.
       - If a label is provided, it will be used in the error message.
       - If no label exists, it will fall back to the field’s name.

2. Remove _index from Paths for Unnamed Fields (In the validationError
component only):
- For unnamed fields like rows and collapsibles, the _index prefix is
now stripped from the output to make it cleaner.
       - Instead of `_index-0.text`, it now outputs just `Text`.

3. Reformat the Error Path for Readability:
- The error message format has been improved to be more human-readable,
showing the field hierarchy in a structured way with array indices
converted to 1-based numbers.

#### Example transformation:

##### Before:
The following fields are invalid: `items.0.subArray.0.text`

##### After:
The following fields are invalid: `Items 1 > SubArray 1 > Text`
2025-01-17 09:42:46 -05:00
Sasha
eff75f9613 fix: deep querying, handle getLocalizedPaths for blocks (#10187)
Fixes https://github.com/payloadcms/payload/issues/10126

Properly handles the `getLocalizedPaths` function for blocks.

Previously, using collection with this config:
```
{
  slug: 'deep-nested',
  fields: [
    {
      type: 'tabs',
      tabs: [
        {
          name: 'content',
          fields: [
            {
              type: 'blocks',
              name: 'blocks',
              blocks: [
                {
                  slug: 'testBlock',
                  fields: [
                    {
                      type: 'tabs',
                      tabs: [
                        {
                          name: 'meta',
                          fields: [
                            {
                              type: 'relationship',
                              relationTo: 'movies',
                              name: 'movie',
                            },
                          ],
                        },
                      ],
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
  ],
},

```
The following query didn't work in MongoDB:
```ts
const result = await payload.find({
  collection: 'deep-nested',
  where: {
    'content.blocks.meta.movie': {
      equals: movie.id,
    },
  },
})
```
2024-12-27 09:46:09 -05:00
Sasha
5753efb0a4 fix(db-mongodb): querying by localized polymorphic relationships using objects (#10037)
Previously, queries like this didn't work:
```ts
const res = await payload.find({
  collection: 'polymorphic-relationships',
  where: {
    polymorphicLocalized: {
      equals: {
        relationTo: 'movies',
        value: movie.id,
      },
    },
  },
})
```

This was due to the incorrectly passed path to MongoDB without
`.{locale}` suffix.
Additionally, to MongoDB now we send:
```

{
  $or: [
    {
      polymorphic: {
        $eq: {
          relationTo: formattedValue.relationTo,
          value: formattedValue.value,
        },
      },
    },
    {
      polymorphic: {
        $eq: {
          relationTo: 'movies',
          value: 'some-id',
        },
      },
    },
  ],
},
```

Instead of:
```
{
  $and: [
    {
      'polymorphic.relationTo': {
        $eq: 'movies ',
      },
    },
    {
      'polymorphic.value': {
        $eq: 'some-id ',
      },
    },
  ],
}
```

To match the _exact_ value. This is essential when we do querying by
relationships with `hasMany: true` and custom IDs that can be repeated.
`$or` is needed if for some reason keys are stored in the DB in a
different order
2024-12-19 10:42:15 -05:00
Sasha
d21fca9156 feat: support relationship writes using objects instead of IDs (#9253)
### What?
Previously, this code led to a validation error because `movie` is an
object and you needed to use `movie.id` instead.
```ts
const movie = await payload.create({ collection: 'movies', data: {} })
const result = await payload.create({
  collection: 'object-writes',
  data: {
    many: [movie],
    manyPoly: [{ relationTo: 'movies', value: movie }],
    one: movie,
    onePoly: {
      relationTo: 'movies',
      value: movie,
    },
  },
})
```
While it's simple to modify this example, it's more painful when you
have a data with `depth` > 0 and then you want to update that document.

### Why?
Better DX as less checks needed, and TypeScript says that we can pass an
object.

### How?
Sanitizes the field value in the root `beforeValidate` hook
2024-11-17 11:25:32 +02:00
Dan Ribbens
7c6f41936b feat(db-mongodb)!: update mongoose to 8.8.1 (#9115)
### What?
Upgrades mongoose from 6 to latest `v8.8.1`

Fixes https://github.com/payloadcms/payload/issues/9171

### Why?
Compatibilty with Mongodb Atlas

### How?
- Updates deps
- Changed ObjectId from bson-objectid to use `new Type.ObjectId` from
mongoose for compatibility (only inside of db-mongodb)
- Internal type adjustments

https://github.com/payloadcms/payload/discussions/9088

BREAKING CHANGES:
All projects with existing data having versions enabled, or relationship or upload fields will want to create the predefined migration that converts all strings to ObjectIDs where needed. This can be created using `payload migrate:create --file @payloadcms/mongodb/relationships-v2-v3`.
For projects making use of the exposed Models from mongoose, review the
upgrade guides from [v6 to
v7](https://mongoosejs.com/docs/7.x/docs/migrating_to_7.html) and [v7 to
v8](https://mongoosejs.com/docs/migrating_to_8.html) and make
adjustments as needed.

---------

Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
2024-11-15 12:03:56 -05:00
Sasha
5b97ac1a67 fix: querying relationships by id path with REST (#9013)
### What?
Fixes the issue with querying by `id` from REST / `overrideAccess:
false`.
For example, this didn't work:

`/api/loans?where[book.bibliography.id][equals]=67224d74257b3f2acddc75f4`
```
QueryError: The following path cannot be queried: id
```

### Why?
We support this syntax within the Local API.

### How?
Now, for simplicity we sanitize everything like
`relation.otherRelation.id` to `relation.otherRelation`

Fixes https://github.com/payloadcms/payload/issues/9008
2024-11-04 17:57:41 +02:00
Dan Ribbens
1bf580fac3 feat: join field works with hasMany relationships (#8493)
Join field works on relationships and uploads having `hasMany: true`

---------

Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
2024-10-08 11:40:34 -04:00
Sasha
265d7fa0e2 fix(drizzle): equals polymorphic querying with object notation (#8316)
Previously, this wasn't valid in Postgres / SQLite:
```ts
const res = await payload.find({
  collection: 'polymorphic-relationships',
  where: {
    polymorphic: {
      equals: {
        relationTo: 'movies',
        value: movie.id,
      },
    },
  },
})
```

Now it works and actually in more performant way than this:
```ts
const res = await payload.find({
  collection: 'polymorphic-relationships',
  where: {
    and: [
      {
        'polymorphic.relationTo': {
          equals: 'movies',
        },
      },
      {
        'polymorphic.value': {
          equals: 'movies',
        },
      },
    ],
  },
})
``` 

Why? Because with the object notation, the output SQL is: `movies_id =
1` - checks exactly 1 column in the `*_rels` table, while with the
separate query by `relationTo` and `value` we need to check against
_each_ possible relationship collection with OR.
2024-09-20 11:18:13 -04:00
Sasha
31ffc57366 fix(drizzle): in query on polymorphic relations across ID types (#8240)
Fixes querying using `in` operator by polymorphic relationship value.
The previous PR https://github.com/payloadcms/payload/pull/8191 didn't
handle the case when the incoming query value is an array and therefore
each item of the array can have a different type.
Ensures test coverage
2024-09-17 12:57:00 -04:00
Sasha
a0a1e20193 fix(drizzle): polymorphic querying of different ID types (#8191)
This PR fixes querying by a relationship field that has custom IDs in
`relationTo` with different types.
Now, in this case, we do cast the ID value in the database.

Example of the config / int test that reproduced the issue:

```ts
{
  slug: 'posts-a',
  fields: [],
},
{
  slug: 'posts-b',
  fields: [],
},
{
  slug: 'posts-custom-id',
  fields: [{ name: 'id', type: 'text' }],
},
{
  slug: 'roots',
  fields: [
    {
      name: 'rel',
      relationTo: ['posts-a', 'posts-b', 'posts-custom-id'],
      type: 'relationship',
    },
  ],
},
```

```ts
const postA = await payload.create({ collection: 'posts-a', data: {} })
const postB = await payload.create({ collection: 'posts-b', data: {} })
const postC = await payload.create({
  collection: 'posts-custom-id',
  data: { id: crypto.randomUUID() },
})

const root_1 = await payload.create({
  collection: 'roots',
  data: {
    rel: {
      value: postC.id,
      relationTo: 'posts-custom-id',
    },
  },
})

const res_1 = await payload.find({
  collection: 'roots',
  where: {
    'rel.value': { equals: postC.id },
  },
})

// COALESCE types integer and character varying cannot be matched

expect(res_1.totalDocs).toBe(1)
```

<!--

For external contributors, please include:

- A summary of the pull request and any related issues it fixes.
- Reasoning for the changes made or any additional context that may be
useful.

Ensure you have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.

 -->
2024-09-16 10:39:55 -04:00
Sasha
0c563ebd73 fix(db-postgres): querying on array wtihin a relationship field (#8152)
## Description

Fixes https://github.com/payloadcms/payload/issues/6037

- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.

## Type of change

<!-- Please delete options that are not relevant. -->

- [x] Bug fix (non-breaking change which fixes an issue)

## Checklist:

- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
2024-09-10 08:44:38 -04:00
Alessio Gravili
90b7b20699 feat!: beta-next (#7620)
This PR makes three major changes to the codebase:

1. [Component Paths](#component-paths)
Instead of importing custom components into your config directly, they
are now defined as file paths and rendered only when needed. That way
the Payload config will be significantly more lightweight, and ensures
that the Payload config is 100% server-only and Node-safe. Related
discussion: https://github.com/payloadcms/payload/discussions/6938

2. [Client Config](#client-config)
Deprecates the component map by merging its logic into the client
config. The main goal of this change is for performance and
simplification. There was no need to deeply iterate over the Payload
config twice, once for the component map, and another for the client
config. Instead, we can do everything in the client config one time.
This has also dramatically simplified the client side prop drilling
through the UI library. Now, all components can share the same client
config which matches the exact shape of their Payload config (with the
exception of non-serializable props and mapped custom components).

3. [Custom client component are no longer
server-rendered](#custom-client-components-are-no-longer-server-rendered)
Previously, custom components would be server-rendered, no matter if
they are server or client components. Now, only server components are
rendered on the server. Client components are automatically detected,
and simply get passed through as `MappedComponent` to be rendered fully
client-side.

## Component Paths

Instead of importing custom components into your config directly, they
are now defined as file paths and rendered only when needed. That way
the Payload config will be significantly more lightweight, and ensures
that the Payload config is 100% server-only and Node-safe. Related
discussion: https://github.com/payloadcms/payload/discussions/6938

In order to reference any custom components in the Payload config, you
now have to specify a string path to the component instead of importing
it.

Old:

```ts
import { MyComponent2} from './MyComponent2.js'

admin: {
  components: {
    Label: MyComponent2
  },
},
```

New:

```ts
admin: {
  components: {
    Label: '/collections/Posts/MyComponent2.js#MyComponent2', // <= has to be a relative path based on a baseDir configured in the Payload config - NOT relative based on the importing file
  },
},
```

### Local API within Next.js routes

Previously, if you used the Payload Local API within Next.js pages, all
the client-side modules are being added to the bundle for that specific
page, even if you only need server-side functionality.

This `/test` route, which uses the Payload local API, was previously 460
kb. It is now down to 91 kb and does not bundle the Payload client-side
admin panel anymore.

All tests done
[here](https://github.com/payloadcms/payload-3.0-demo/tree/feat/path-test)
with beta.67/PR, db-mongodb and default richtext-lexical:

**dev /admin before:**
![CleanShot 2024-07-29 at 22 49
12@2x](https://github.com/user-attachments/assets/4428e766-b368-4bcf-8c18-d0187ab64f3e)

**dev /admin after:**
![CleanShot 2024-07-29 at 22 50
49@2x](https://github.com/user-attachments/assets/f494c848-7247-4b02-a650-a3fab4000de6)

---

**dev /test before:**
![CleanShot 2024-07-29 at 22 56
18@2x](https://github.com/user-attachments/assets/1a7e9500-b859-4761-bf63-abbcdac6f8d6)

**dev /test after:**
![CleanShot 2024-07-29 at 22 47
45@2x](https://github.com/user-attachments/assets/f89aa76d-f2d5-4572-9753-2267f034a45a)

---

**build before:**
![CleanShot 2024-07-29 at 22 57
14@2x](https://github.com/user-attachments/assets/5f8f7281-2a4a-40a5-a788-c30ddcdd51b5)

**build after::**
![CleanShot 2024-07-29 at 22 56
39@2x](https://github.com/user-attachments/assets/ea8772fd-512f-4db0-9a81-4b014715a1b7)

### Usage of the Payload Local API / config outside of Next.js

This will make it a lot easier to use the Payload config / local API in
other, server-side contexts. Previously, you might encounter errors due
to client files (like .scss files) not being allowed to be imported.

## Client Config

Deprecates the component map by merging its logic into the client
config. The main goal of this change is for performance and
simplification. There was no need to deeply iterate over the Payload
config twice, once for the component map, and another for the client
config. Instead, we can do everything in the client config one time.
This has also dramatically simplified the client side prop drilling
through the UI library. Now, all components can share the same client
config which matches the exact shape of their Payload config (with the
exception of non-serializable props and mapped custom components).

This is breaking change. The `useComponentMap` hook no longer exists,
and most component props have changed (for the better):

```ts
const { componentMap } = useComponentMap() // old
const { config } = useConfig() // new
```

The `useConfig` hook has also changed in shape, `config` is now a
property _within_ the context obj:

```ts
const config = useConfig() // old
const { config } = useConfig() // new
```

## Custom Client Components are no longer server rendered

Previously, custom components would be server-rendered, no matter if
they are server or client components. Now, only server components are
rendered on the server. Client components are automatically detected,
and simply get passed through as `MappedComponent` to be rendered fully
client-side.

The benefit of this change:

Custom client components can now receive props. Previously, the only way
for them to receive dynamic props from a parent client component was to
use hooks, e.g. `useFieldProps()`. Now, we do have the option of passing
in props to the custom components directly, if they are client
components. This will be simpler than having to look for the correct
hook.

This makes rendering them on the client a little bit more complex, as
you now have to check if that component is a server component (=>
already has been rendered) or a client component (=> not rendered yet,
has to be rendered here). However, this added complexity has been
alleviated through the easy-to-use `<RenderMappedComponent />` helper.

This helper now also handles rendering arrays of custom components (e.g.
beforeList, beforeLogin ...), which actually makes rendering custom
components easier in some cases.

## Misc improvements

This PR includes misc, breaking changes. For example, we previously
allowed unions between components and config object for the same
property. E.g. for the custom view property, you were allowed to pass in
a custom component or an object with other properties, alongside a
custom component.

Those union types are now gone. You can now either pass an object, or a
component. The previous `{ View: MyViewComponent}` is now `{ View: {
Component: MyViewComponent} }` or `{ View: { Default: { Component:
MyViewComponent} } }`.

This dramatically simplifies the way we read & process those properties,
especially in buildComponentMap. We can now simply check for the
existence of one specific property, which always has to be a component,
instead of running cursed runtime checks on a shared union property
which could contain a component, but could also contain functions or
objects.

![CleanShot 2024-07-29 at 23 07
07@2x](https://github.com/user-attachments/assets/1e75aa4c-7a4c-419f-9070-216bb7b9a5e5)

![CleanShot 2024-07-29 at 23 09
40@2x](https://github.com/user-attachments/assets/b4c96450-6b7e-496c-a4f7-59126bfd0991)

- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.

---------

Co-authored-by: PatrikKozak <patrik@payloadcms.com>
Co-authored-by: Paul <paul@payloadcms.com>
Co-authored-by: Paul Popus <paul@nouance.io>
Co-authored-by: Jacob Fletcher <jacobsfletch@gmail.com>
Co-authored-by: James <james@trbl.design>
2024-08-13 12:54:33 -04:00
Alessio Gravili
83fd4c6622 chore: run lint and prettier on entire codebase 2024-07-11 15:27:01 -04:00
Jarrod Flesch
0711f880ff chore!: simplify api handler (#6910)
Removes PayloadRequestWithData in favour of just PayloadRequest with
optional types for `data` and `locale`

`addDataAndFileToRequest` and `addLocalesToRequestFromData` now takes in
a single argument instead of an object

```ts
// before
await addDataAndFileToRequest({ request: req })
addLocalesToRequestFromData({ request: req })

// current
await addDataAndFileToRequest(req)
addLocalesToRequestFromData(req)
```

---------

Co-authored-by: Paul Popus <paul@nouance.io>
2024-07-02 09:47:03 -04:00
Jacob Fletcher
9e76c8f4e3 feat!: prebundle payload, ui, richtext-lexical (#6579)
# Breaking Changes

### New file import locations

Exports from the `payload` package have been _significantly_ cleaned up.
Now, just about everything is able to be imported from `payload`
directly, rather than an assortment of subpath exports. This means that
things like `import { buildConfig } from 'payload/config'` are now just
imported via `import { buildConfig } from 'payload'`. The mental model
is significantly simpler for developers, but you might need to update
some of your imports.

Payload now exposes only three exports:

1. `payload` - all types and server-only Payload code
2. `payload/shared` - utilities that can be used in either the browser
or in Node environments
3. `payload/node` - heavy utilities that should only be imported in Node
scripts and never be imported into bundled code like Next.js

### UI library pre-bundling

With this release, we've dramatically sped up the compile time for
Payload by pre-bundling our entire UI package for use inside of the
Payload admin itself. There are new exports that should be used within
Payload custom components:

1. `@payloadcms/ui/client` - all client components 
2. `@payloadcms/ui/server` - all server components

For all of your custom Payload admin UI components, you should be
importing from one of these two pre-compiled barrel files rather than
importing from the more deeply nested exports directly. That will keep
compile times nice and speedy, and will also make sure that the bundled
JS for your admin UI is kept small.

For example, whereas before, if you imported the Payload `Button`, you
would have imported it like this:

```ts
import { Button } from '@payloadcms/ui/elements/Button'
```

Now, you would import it like this:

```ts
import { Button } from '@payloadcms/ui/client'
```

This is a significant DX / performance optimization that we're pretty
pumped about.

However, if you are importing or re-using Payload UI components
_outside_ of the Payload admin UI, for example in your own frontend
apps, you can import from the individual component exports which will
make sure that the bundled JS is kept to a minimum in your frontend
apps. So in your own frontend, you can continue to import directly to
the components that you want to consume rather than importing from the
pre-compiled barrel files.

Individual component exports will now come with their corresponding CSS
and everything will work perfectly as-expected.

### Specific exports have changed

- `'@payloadcms/ui/templates/Default'` and
`'@payloadcms/ui/templates/Minimal`' are now exported from
`'@payloadcms/next/templates'`
- Old: `import { LogOut } from '@payloadcms/ui/icons/LogOut'` new:
`import { LogOutIcon } from '@payloadcms/ui/icons/LogOut'`

## Background info

In effort to make local dev as fast as possible, we need to import as
few files as possible so that the compiler has less to process. One way
we've achieved this in the Admin Panel was to _remove_ all .scss imports
from all components in the `@payloadcms/ui` module using a build
process. This stripped all `import './index.scss'` statements out of
each component before injecting them into `dist`. Instead, it bundles
all of the CSS into a single `main.css` file, and we import _that_ at
the root of the app.

While this concept is _still_ the right solution to the problem, this
particular approach is not viable when using these components outside
the Admin Panel, where not only does this root stylesheet not exist, but
where it would also bloat your app with unused styles. Instead, we need
to _keep_ these .scss imports in place so they are imported directly
alongside your components, as expected. Then, we need create a _new_
build step that _separately_ compiles the components _without_ their
stylesheets—this way your app can consume either as needed from the new
`client` and `server` barrel files within `@payloadcms/ui`, i.e. from
within `@payloadcms/next` and all other admin-specific packages and
plugins.

This way, all other applications will simply import using the direct
file paths, just as they did before. Except now they come with
stylesheets.

And we've gotten a pretty awesome initial compilation performance boost.

---------

Co-authored-by: James <james@trbl.design>
Co-authored-by: Alessio Gravili <alessio@gravili.de>
2024-06-17 14:25:36 -04:00
Jarrod Flesch
22c53392a3 chore: improves types for payloadRequest (#6012) 2024-04-25 10:23:03 -04:00
Paul
e64660f2d8 fix(db-postgres): querying by localised relations postgres (#5686) 2024-04-05 10:37:58 -04:00
Dan Ribbens
a1e8c4eb2b fix(db-postgres): query with contains operator hasMany (#5481) 2024-03-26 15:26:46 -04:00
Dan Ribbens
0ce752af79 fix: regression of filterOptions using different transaction (#5450) 2024-03-26 10:13:57 -04:00
Elliot DeNolf
c1e1cd0596 test: misc test dir type fixes 2024-03-21 11:51:29 -04:00
Elliot DeNolf
c5ecf48d94 chore: add test/ to workspace, update most references 2024-03-19 00:59:56 -04:00
Elliot DeNolf
14eb66c87d test: refactor int tests to use initPayloadInt which reduces boilerplate 2024-03-16 10:11:00 -04:00
Alessio Gravili
6789e61488 chore: run lint & prettier on everything 2024-03-14 23:53:47 -04:00
Dan Ribbens
d193c677c7 chore: attach mongoMemoryServer to db and destroy in tests (#5326)
* chore: attach mongoMemoryServer to db and destroy in tests

* bump mongodb-memory-server to 9.x

---------

Co-authored-by: Paul Popus <paul@nouance.io>
2024-03-14 15:41:20 -04:00
Dan Ribbens
09484667f0 chore: relationships int test race condition 2024-03-12 16:30:06 -04:00
Jarrod Flesch
0e91cddab9 chore: fix relationships test 2024-03-08 16:41:03 -05:00
Alessio Gravili
8895f6420f chore: fix all esm test suite imports 2024-03-08 14:42:24 -05:00
Dan Ribbens
b9dec2f714 Chore/next poc merge main (#5204)
* wip moves payload, user and data into partial req

* chore: adjust req type

* chore(next): installs sass and resolves type errors

* feat: working login route/view

* fix: me route

* chore(next): scaffolds access routes (#4562)

* chore(next): scaffolds admin layout and dashboard view (#4566)

* chore(next): builds initPage utility (#4589)

* feat(3.0): next route handlers (#4590)

* chore: removes old files

* chore(next): ssr list view (#4594)

* chore: removes old files

* chore: adjusts graphql file imports to align with new operation exports

* chore: allows for custom endpoints

* chore: cleanup

* chore(next): ssr edit view (#4614)

* chore(ui): ssr main nav (#4619)

* chore(next): ssr account view (#4620)

* chore(next): ssr auth views and document create (#4631)

* chore(next): ssr globals view (#4640)

* chore(next): scaffolds document layout (#4644)

* chore(next): ssr versions view (#4645)

* chore(next): ssr field conditions (#4675)

* chore(next): ssr field validations (#4700)

* chore(next): moves dashboard view into next dir

* chore(next): moves account view into next dir

* chore(next): moves global edit view into next dir

* chore(next): returns isolated configs and locale from initPage

* chore(next): ssr api view (#4721)

* feat: adds i18n functionality within Rest API, Local and Client contexts (#4749)

* chore: separate client translation groups with empty line

* chore: add missing translation used in db adapters

* chore: simplify next/routes export and import paths

* chore: renames PayloadT to Payload

* chore(next): custom views (#4748)

* chore: fix translation tsconfig

* chore: adjust other package ts-configs that rely on translations

* chore(next): installs @payloadcms/ui as direct dependency

* chore(next): progress to build

* chore(next): migrates types (#4792)

* fixes acccept-language detection

* chore(next): moves remaining components out from payload core (#4794)

* chore(deps): removes all unused dependencies from payload core (#4797)

* chore(next): achieves buildable state (#4803)

* adds Translation component and removes more react-i18next

* fixes up remaining translation strings

* fixes a few i18n TODO's

* chore: remaining translation strings without colons

* chore: adds missing ja translations

* chore(next): ssr group field (#4830)

* chore: removes placeholder t function

* chore: removes old file

* chore(bundler-webpack): removes webpack bundler

* chore(bundler-vite): removes vite bundler

* chore(next): ssr tabs field (#4863)

* chore(next): ssr row field

* chore(next): ssr textarea field

* chore(next): wires server action into document edit view (#4873)

* chore(next): conditional logic (#4880)

* chore(next): ssr radio, point, code, json, ui, and hidden fields (#4891)

* chore(next): ssr collapsible field (#4894)

* chore: remove findByID from req

* chore: adjusts file property on request type

* comment clarification

* chore: wires up busboy with Requst readstream

* chore: ports over express-fileupload into a NextJS compatible format

* chore: adjust upload file structure

* chore: adds try/catch around routes, corrects a few route responses

* chore: renames file/function

* chore: improve req type safety in local operations, misc req.files replacements

* chore: misc type and fn export changes

* chore: ensures root routes take pass unmodified request to root routes

* chore: improve types

* chore: consolidates locale api req initialization (#4922)

* chore(next): overhauls field rendering strategy (#4924)

* chore(next): ssr array field (#4937)

* chore(next): ssr blocks field (#4942)

* chore(next): ssr upload field and document drawer (#4957)

* chore(next): wires form submissions (#4982)

* chore: api handler adjustments

* feat: adds graphql playground handler

* adds credentials include setting to playground

* remove old playground init, stub graphql handler location

* fix: allow for null fallbackLocale

* fix: correctly prioritize locales passed as null

* chore: move all graphql code into next package

* graphql changes

* chore: semi working version of graphql http layer

* gql fix attempts

* rm console log

* chore: partial gql changes

* chore: adds gql and gql-http back into payload

* chore: removes collection from req

* chore: separates graphql package out for schema generation

* chore: dep cleanup

* chore: move graphql handlers

* chore: removes unused deps

* chore(next): ssr list view (#5032)

* chore: refactor response handler order for custom endpoints

* chore: add back in condition for collection GET path with 2 slugs

* chore: rm optional chain

* chore: import sort route file

* chore: allows custom endpoints to attempt before erroring

* feat: adds memoization to translation functions (#5036)

* chore: fix APIError import

* chore: return attemptCustomEndpointBeforeError responses

* chore(next): properly instantiates table columns

* fix(next): attaches params to req and properly assigns prefs key (#5042)

* chore: reorganize next route order

* chore(next): adds RouteError handler to next routes

* chore: builds payload successfully

* chore: misc file omissions

* fix(ui): maintains proper column order

* fix(ui): ensures first cell is a link

* fix(next): properly copies url object in createPayloadRequest (#5064)

* fix(ui): bumps react-toastify to v10.0.4 to fix hydration warnings

* feat: add route for static file GET requests (#5065)

* chore(next): allows resolved config promise to be thread through initPage (#5071)

* chore(ui): conditionally renders field label from props

* feat(next): next install script

* chore: pass config to route handlers

* feat: initial test suite framework (#4929)

* chore(next): renderable account, api, and create first user views (#5084)

* fix(next): properly parses search params in find, update, and delete handlers (#5088)

* chore(next): ssr versions view (#5085)

* chore: adds homepage for scss testing

* chore: moves dev folder to top, establishes new test pattern

* chore: working turbopack

* chore: sets up working dynamic payload-config imports

* remove unused code

* chore: rm console log

* misc

* feat: correctly subs out ability to boot REST API within same process

* chore: WIP dev suites

* chore: removes need for REST_API folder in test dir

* removes duplicate bootAdminPanel fn

* misc

* specify default export

* chore: sets up jest to work with next/jest

* chore: progress to mongodb and sharp builds

* chore: passing community tests

* chore: sorta workin

* chore: adjust payload-config import

* chore: adds rest client for Next handlers

* chore: removes test garb

* chore: restores payload-config tsconfig path temporarily

* chore: establishes pattern for memory db during tests

* chore: bumps mongoose to 7

* chore(next): 404s on nested create urls

* chore: functional _community e2e

* chore: increases e2e expect timeout

* fix(next): sanitizes locale toString from client config

* chore: type fixes

* chore: pulls mongodb from main

* chore: uses graphql to log user in

* feat: passing auth test suite

* chore(ui): threads params through context and conditionally renders document tabs (#5094)

* feat(ui): adds params context (#5095)

* chore: removes unecessary memory allocation for urlPropertiesObject object

* chore: passing graphql test suite

* chore: removes references to bson

* chore: re-enables mongodb memory server for auth test suite

* chore: replace bson with bson-objectid

* feat: passing collections-rest int suite

* chore: fixes bad imports

* chore: more passing int suites

* feat: passing globals int tests

* feat: passing hooks int test suite

* chore: remove last express file

* chore: start live-preview int test migration

* chore: passing localization int tests

* passing relationships int tests

* chore: partial passing upload int tests

* chore: fixes scss imports

* chore(ui): renders document info provider at root (#5106)

* chore: adds schema path to useFieldPath provider, more passing tests

* chore: begins work to optimize translation imports

* chore: add translations to ui ts-config references

* chore: add exports folder to package json exports

* chore: adds readme how-to-use instructions

* chore: attempts refactor of translation imports

* chore: adds authentication:account translation key to server keys

* chore: finishes translation optimization

* chore: ignores warnings from mongodb

* chore(ui): renders live document title (#5115)

* chore(ui): ssr document tabs (#5116)

* chore: handles redirecting from login

* chore: handle redirect with no searchParams

* chore: handle missing segments

* chore(next): migrates server action into standalone api endpoint (#5122)

* chore: adjust dashboard colection segments

* test: update e2e suites

* fix(ui): prevents unnecessary calls to form state

* chore: fix finding global config fields from schema path

* fix(next): executes root POST endpoints

* chore(ui): ignores values returned by form state polling

* chore: scaffolds ssr rte

* chore: renders client leaves

* chore: server-side rendered rich text elements

* chore: defines ClientFunction pattern

* chore(ui): migrates relationship field

* chore: adds translations, cleans up slate

* chore: functional slate link

* chore: slate upload ssr

* chore: relationship slate ssr

* chore: remaining slate ssr

* chore: fixes circular workspace dep

* chore: correct broken int test import paths

* chore: remove media files from root

* chore: server renders custom edit view

* fix(ui): resolves infinite loading in versions view

* fix(next): resolves global edit view lookup

* chore: payload builds

* chore: delete unused files

* chore: removes local property from payload

* chore: adds mongodb as dev dep in db-mongodb package

* chore: hide deprecation warnings for tempfile and jest-environment-jsdom

* chore: remove all translations from translations dist

* chore: clean ts-config files

* chore: simple type fixes

* chore(ui): server renders custom list view

* chore: fix next config payload-config alias

* chore: adds turbo alias paths

* chore: adjusts translation generation

* chore: improve auth function

* chore: eslint config for packages/ui

* chore(ui): exports FormState

* chore(next): migrates account view to latest patterns

* chore: disable barbie mode

* chore(ui): lints

* chore(next): lints

* chore: for alexical

* chore: custom handler type signature adjustment

* fix: non-boolean condition result causes infinite looping (#4579)

* chore(richtext-lexical): upgrade lexical from v0.12.5 to v0.12.6 (#4732)

* chore(richtext-lexical): upgrade all lexical packages from 0.12.5 to 0.12.6

* fix(richtext-lexical): fix TypeScript errors

* fix indenting

* feat(richtext-lexical): Blocks: generate type definitions for blocks fields (#4529)

* feat(richtext-lexical)!: Update lexical from 0.12.6 to 0.13.1, port over all useful changes from playground (#5066)

* feat(richtext-lexical): Update lexical from 0.12.6 to 0.13.1, port over all useful changes from playground

* chore: upgrade lexical version used in monorepo

* chore: remove the 3

* chore: upgrade nodemon versions (#5059)

* feat: add more options to addFieldStatePromise so that it can be used for field flattening (#4799)

* feat(plugin-seo)!: remove support for payload <2.7.0 (#4765)

* chore(plugin-seo): remove test script from package.json (#4762)

* chore: upgrade @types/nodemailer from v6.4.8 to v6.4.14 (#4733)

* chore: revert auth and initPage changes

* chore(next): moves edit and list views (#5170)

* fix: "The punycode module is deprecated" warning by updating nodemailer

* chore: adjust translations tsconfig paths in root

* chore: fix merge build

---------

Co-authored-by: Jarrod Flesch <jarrodmflesch@gmail.com>
Co-authored-by: Jacob Fletcher <jacobsfletch@gmail.com>
Co-authored-by: Jarrod Flesch <30633324+JarrodMFlesch@users.noreply.github.com>
Co-authored-by: Elliot DeNolf <denolfe@gmail.com>
Co-authored-by: James <james@trbl.design>
Co-authored-by: Alessio Gravili <alessio@gravili.de>
Co-authored-by: Alessio Gravili <70709113+AlessioGr@users.noreply.github.com>
2024-02-28 13:44:17 -05:00
Jarrod Flesch
19a4a99e76 chore: partial passing upload int tests 2024-02-19 10:03:35 -05:00
Jarrod Flesch
1c9ba5b512 passing relationships int tests 2024-02-17 01:00:48 -05:00
Jarrod Flesch
717a6b6d07 feat: initial test suite framework (#4929) 2024-02-14 09:46:11 -05:00
Jarrod Flesch
fe888b5f6c fix: query validation on relationship fields (#4353) 2023-12-01 16:03:58 -05:00
Dan Ribbens
6ea29094ba fix(db-postgres): incorrect pagination totalDocs (#4248)
Co-authored-by: Alessio Gravili <alessio@bonfireleads.com>
2023-11-22 16:14:14 -05:00
Take Weiland
acad2888cd fix: fixes creation of related documents within a transaction if filterOptions is used (#4087) 2023-11-10 10:37:58 -05:00