Commit Graph

610 Commits

Author SHA1 Message Date
Sasha
840dde2b17 fix(db-mongodb): bump mongoose to 8.8.3 (#9747)
Fixes https://github.com/payloadcms/payload/issues/9729. The current
version has vulnerability
https://avd.aquasec.com/nvd/2024/cve-2024-53900/. Technically, Payload
doesn't use described in the report
[`$where`](https://www.mongodb.com/docs/manual/reference/operator/query/where/#op._S_where)
property in its queries at all, but it may affect those who access
mongoose via `payload.db.collections` directly
2024-12-05 18:43:14 +02:00
Elliot DeNolf
0ca473e6ca chore(release): v3.4.0 [skip ci] 2024-12-04 10:32:41 -05:00
Elliot DeNolf
a6ba9e3a41 chore(release): v3.3.0 [skip ci] 2024-12-03 12:53:42 -05:00
Dan Ribbens
631edd4c17 fix: latest: true version disappear on parallel writes (#9032)
What?
Fixes issue when on parallel writes in result you can have 0 latest:
true versions.

Why?
There must be always a version with latest: true

How?
Ensures that we always have a version with latest: true by adding a
filter on createdAt < createdVersion.createdAt.
Instead, this ponentially can lead to a situation where we have 2
versions with latest: true, if they were created at the exact same time,
but this shouldn't happen in a real world scenario and it's much less
problematic than not having a version with latest: true.

Fixes https://github.com/payloadcms/payload/issues/5895

Changes from #8986

---------

Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
2024-12-02 14:34:45 -05:00
Elliot DeNolf
07e40d37ac chore(release): v3.2.2 [skip ci] 2024-11-29 12:59:59 -05:00
Elliot DeNolf
be59d52ac7 chore(release): v3.2.1 [skip ci] 2024-11-26 22:29:29 -05:00
Elliot DeNolf
c8bee29920 chore(release): v3.2.0 [skip ci] 2024-11-26 20:39:41 -05:00
Elliot DeNolf
058bd02ebd chore(release): v3.1.1 [skip ci] 2024-11-25 16:42:21 -05:00
Sasha
b5f89d5199 fix(db-mongodb): sanitizeRelationshipIDs named tabs within tabs (#9400)
Ensures `sanitizeRelationshipIDs` works properly in any case
Updates predefinedMigration to work with new globals
Skips ObjectID creation errors to not fail with outdated data to the
schema.
2024-11-25 16:27:47 -05:00
Sasha
cae300e8e3 perf: flattenedFields collection/global property, remove deep copying in validateQueryPaths (#9299)
### What?
Improves querying performance of the Local API, reduces copying overhead

### How?

Adds `flattenedFields` property for sanitized collection/global config
which contains fields in database schema structure.
For example, It removes rows / collapsible / unnamed tabs and merges
them to the parent `fields`, ignores UI fields, named tabs are added as
`type: 'tab'`.
This simplifies code in places like Drizzle `transform/traverseFields`.
Also, now we can avoid calling `flattenTopLevelFields` which adds some
overhead and use `collection.flattenedFields` / `field.flattenedFields`.

By refactoring `configToJSONSchema.ts` with `flattenedFields` it also
1. Fixes https://github.com/payloadcms/payload/issues/9467
2. Fixes types for UI fields were generated for `select`



Removes this deep copying for each `where` query constraint
58ac784425/packages/payload/src/database/queryValidation/validateQueryPaths.ts (L69-L73)
which potentially can add overhead if you have a large collection/global
config

UPD:
The overhead is even much more than in the benchmark below if you have
Lexical with blocks.

Benchmark in `relationships/int.spec.ts`:
```ts
const now = Date.now()
for (let i = 0; i < 10; i++) {
  const now = Date.now()
  for (let i = 0; i < 300; i++) {
    const query = await payload.find({
      collection: 'chained',
      where: {
        'relation.relation.name': {
          equals: 'third',
        },
        and: [
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
        ],
      },
    })
  }

  payload.logger.info(`#${i + 1} ${Date.now() - now}`)
}
payload.logger.info(`Total ${Date.now() - now}`)
```

Before:
```
[02:11:48] INFO: #1 3682
[02:11:50] INFO: #2 2199
[02:11:54] INFO: #3 3483
[02:11:56] INFO: #4 2516
[02:11:59] INFO: #5 2467
[02:12:01] INFO: #6 1987
[02:12:03] INFO: #7 1986
[02:12:05] INFO: #8 2375
[02:12:07] INFO: #9 2040
[02:12:09] INFO: #10 1920
    [PASS] Relationships > Querying > Nested Querying > should allow querying two levels deep (24667ms)
[02:12:09] INFO: Total 24657
```

After:
```
[02:12:36] INFO: #1 2113
[02:12:38] INFO: #2 1854
[02:12:40] INFO: #3 1700
[02:12:42] INFO: #4 1797
[02:12:44] INFO: #5 2121
[02:12:46] INFO: #6 1774
[02:12:47] INFO: #7 1670
[02:12:49] INFO: #8 1610
[02:12:50] INFO: #9 1596
[02:12:52] INFO: #10 1576
    [PASS] Relationships > Querying > Nested Querying > should allow querying two levels deep (17828ms)
[02:12:52] INFO: Total 17818
```
2024-11-25 10:28:07 -05:00
Elliot DeNolf
65ac739da9 chore(release): v3.1.0 [skip ci] 2024-11-22 11:54:08 -05:00
Jessica Chowdhury
fb3242df0a feat: add skip and force accept flags to migration commands (#8843)
1. Adds flag `--skip-empty` to `migrate:create` to bypass the empty
migration file prompt.
    - Blank migration file will not be created if this flag is passed.
3. Adds flag `--force-accept-warning` to `migrate:fresh` to bypass the
drop database prompt
2024-11-22 10:37:54 -05:00
Elliot DeNolf
025d917fa0 chore(release): v3.0.2 [skip ci] 2024-11-20 16:06:04 -05:00
Elliot DeNolf
9ea26638e9 chore(release): v3.0.1 [skip ci] 2024-11-19 14:33:41 -05:00
Elliot DeNolf
6407e577d3 chore(release): v3.0.0 [skip ci] 2024-11-18 20:10:16 -05:00
Elliot DeNolf
fade739f77 chore(release): v3.0.0-beta.135 [skip ci] 2024-11-18 19:19:55 -05:00
Sasha
665b3536d3 fix(db-mongodb): potential errors in sanitizeRelationshipIDs with ref being a non object (#9292)
### What?
Fixes potential errors when passed to `sanitizeRelationships` `ref`
could potentially be a non object (for example `string`) because of
having in the database data in old structure.
```
"Cannot create property 'a' on string 'B'",
``` 

### Why?
Necessary particularly for the migration script, as it migrates
everything including versions that can have outdated data.

### How?
Ensures passed `ref` is an `object`.
2024-11-18 16:10:13 +02:00
Elliot DeNolf
cb6ceaec76 chore(release): v3.0.0-beta.134 [skip ci] 2024-11-17 11:43:43 -05:00
Elliot DeNolf
457be31ed2 chore(release): v3.0.0-beta.133 [skip ci] 2024-11-16 13:59:41 -05:00
Elliot DeNolf
ff8e7bb968 chore(release): v3.0.0-beta.132 [skip ci] 2024-11-15 18:13:02 -05:00
Elliot DeNolf
90686fa50a chore(release): v3.0.0-beta.131 [skip ci] 2024-11-15 15:34:50 -05:00
Dan Ribbens
7c6f41936b feat(db-mongodb)!: update mongoose to 8.8.1 (#9115)
### What?
Upgrades mongoose from 6 to latest `v8.8.1`

Fixes https://github.com/payloadcms/payload/issues/9171

### Why?
Compatibilty with Mongodb Atlas

### How?
- Updates deps
- Changed ObjectId from bson-objectid to use `new Type.ObjectId` from
mongoose for compatibility (only inside of db-mongodb)
- Internal type adjustments

https://github.com/payloadcms/payload/discussions/9088

BREAKING CHANGES:
All projects with existing data having versions enabled, or relationship or upload fields will want to create the predefined migration that converts all strings to ObjectIDs where needed. This can be created using `payload migrate:create --file @payloadcms/mongodb/relationships-v2-v3`.
For projects making use of the exposed Models from mongoose, review the
upgrade guides from [v6 to
v7](https://mongoosejs.com/docs/7.x/docs/migrating_to_7.html) and [v7 to
v8](https://mongoosejs.com/docs/migrating_to_8.html) and make
adjustments as needed.

---------

Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
2024-11-15 12:03:56 -05:00
Francisco Lourenço
2d2d020c29 feat(db-mongodb): support query options in db update operations (#9191)
The mongodb adapter `updateOne` method accepts an `options` argument
that allows query options to be passed to mongoose. This parameter was
added in https://github.com/payloadcms/payload/pull/8397 to support the
`upsert` operation.

This `options` parameter can also be useful when using the database
adaptor directly without going through the local api. It is true that
the Mongoose models could be used directly in such situations, but the
adapter methods include a lot of useful functionality, like for instance
the sanitization of document and relationship ids, so it is desirable to
be able to use the adapter functions while still being able to provide
mongoose query options (e.g. `{timestamps: false}`).

This PR adds the same options parameter to the other update methods of
the mongodb adapter.
2024-11-14 15:15:03 -05:00
Elliot DeNolf
5ff1bb366c chore: misc cleanup (#9206)
- Proper error logger usage
- Some no-fallthrough warning cleanup
2024-11-14 11:14:08 -05:00
Elliot DeNolf
6baff8a3ba chore(release): v3.0.0-beta.130 [skip ci] 2024-11-13 14:18:00 -05:00
Elliot DeNolf
8c2fc71149 chore(release): v3.0.0-beta.129 [skip ci] 2024-11-12 17:07:25 -05:00
Elliot DeNolf
d3cd9baa9b chore(release): v3.0.0-beta.128 [skip ci] 2024-11-12 15:52:52 -05:00
James Mikrut
6bb4067bb3 feat: adds option to mongoose to ensure indexes (#9155)
Adds option `ensureIndexes` to Mongoose adapter, which will ensure
indexes are ready prior to completing connection.
2024-11-12 14:42:25 -05:00
Tobias Odendahl
09c41d5c86 fix(db-mongodb)!: use dbName for mongodb model (#9107)
### What?
Uses the `collection.dbName` property for the Mongoose model, if
defined.

### Why?
Currently, `collection.dbName` is used for the version name but not for
the actual collection name. Additionally, `autoPluralization` modifies
the `dbName` regardless. This behavior is inconsistent and contradicts
the documentation.

### How?
- Utilize `collection.dbName` instead of `collection.slug`.
- Disable `autoPluralization` for collections with a defined `dbName`.

Related: https://github.com/payloadcms/payload/discussions/9058

**BREAKING CHANGES**
If a `dbName` was previously provided, it will now be used as the
MongoDB collection name instead of the collection `slug`.
`autoPluralization` will not be applied to `dbName`.

---------

Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2024-11-12 13:31:23 -05:00
Alessio Gravili
4f6651433c chore: ensure all packages have consistent licenses and package.json metadata (#9079) 2024-11-12 10:27:36 -05:00
Alessio Gravili
03291472d6 chore: bump all eslint dependencies, run lint and prettier (#9128)
This fixes a peer dependency error in our monorepo, as
eslint-plugin-jsx-a11y finally supports eslint v9.

Additionally, this officially adds TypeScript 5.6 support for
typescript-eslint.
2024-11-12 10:18:22 -05:00
Alessio Gravili
6899a3cc27 fix(db-mongodb): destructuring error when trying to filter date fields by string query (#9116)
Previously, when filtering the internal link relationship in lexical by
typing in the relationship field, it would throw an error, as that
relationship field has a relation to "date-fields".
2024-11-11 20:41:02 -07:00
Jacob Fletcher
c96fa613bc feat!: on demand rsc (#8364)
Currently, Payload renders all custom components on initial compile of
the admin panel. This is problematic for two key reasons:
1. Custom components do not receive contextual data, i.e. fields do not
receive their field data, edit views do not receive their document data,
etc.
2. Components are unnecessarily rendered before they are used

This was initially required to support React Server Components within
the Payload Admin Panel for two key reasons:
1. Fields can be dynamically rendered within arrays, blocks, etc.
2. Documents can be recursively rendered within a "drawer" UI, i.e.
relationship fields
3. Payload supports server/client component composition 

In order to achieve this, components need to be rendered on the server
and passed as "slots" to the client. Currently, the pattern for this is
to render custom server components in the "client config". Then when a
view or field is needed to be rendered, we first check the client config
for a "pre-rendered" component, otherwise render our client-side
fallback component.

But for the reasons listed above, this pattern doesn't exactly make
custom server components very useful within the Payload Admin Panel,
which is where this PR comes in. Now, instead of pre-rendering all
components on initial compile, we're able to render custom components
_on demand_, only as they are needed.

To achieve this, we've established [this
pattern](https://github.com/payloadcms/payload/pull/8481) of React
Server Functions in the Payload Admin Panel. With Server Functions, we
can iterate the Payload Config and return JSX through React's
`text/x-component` content-type. This means we're able to pass
contextual props to custom components, such as data for fields and
views.

## Breaking Changes

1. Add the following to your root layout file, typically located at
`(app)/(payload)/layout.tsx`:

    ```diff
    /* THIS FILE WAS GENERATED AUTOMATICALLY BY PAYLOAD. */
    /* DO NOT MODIFY IT BECAUSE IT COULD BE REWRITTEN AT ANY TIME. */
    + import type { ServerFunctionClient } from 'payload'

    import config from '@payload-config'
    import { RootLayout } from '@payloadcms/next/layouts'
    import { handleServerFunctions } from '@payloadcms/next/utilities'
    import React from 'react'

    import { importMap } from './admin/importMap.js'
    import './custom.scss'

    type Args = {
      children: React.ReactNode
    }

+ const serverFunctions: ServerFunctionClient = async function (args) {
    +  'use server'
    +  return handleServerFunctions({
    +    ...args,
    +    config,
    +    importMap,
    +  })
    + }

    const Layout = ({ children }: Args) => (
      <RootLayout
        config={config}
        importMap={importMap}
    +  serverFunctions={serverFunctions}
      >
        {children}
      </RootLayout>
    )

    export default Layout
    ```

2. If you were previously posting to the `/api/form-state` endpoint, it
no longer exists. Instead, you'll need to invoke the `form-state` Server
Function, which can be done through the _new_ `getFormState` utility:

    ```diff
    - import { getFormState } from '@payloadcms/ui'
    - const { state } = await getFormState({
    -   apiRoute: '',
    -   body: {
    -     // ...
    -   },
    -   serverURL: ''
    - })

    + const { getFormState } = useServerFunctions()
    +
    + const { state } = await getFormState({
    +   // ...
    + })
    ```

## Breaking Changes

```diff
- useFieldProps()
- useCellProps()
```

More details coming soon.

---------

Co-authored-by: Alessio Gravili <alessio@gravili.de>
Co-authored-by: Jarrod Flesch <jarrodmflesch@gmail.com>
Co-authored-by: James <james@trbl.design>
2024-11-11 13:59:05 -05:00
Elliot DeNolf
0c19afcf91 chore(release): v3.0.0-beta.127 [skip ci] 2024-11-11 09:56:14 -05:00
Sasha
0a15388edb feat(db-postgres): add point field support (#9078)
### What?
Adds full support for the point field to Postgres and Vercel Postgres
adapters through the Postgis extension. Fully the same API as with
MongoDB, including support for `near`, `within` and `intersects`
operators.

Additionally, exposes to adapter args:
*
`tablesFilter`https://orm.drizzle.team/docs/drizzle-kit-push#including-tables-schemas-and-extensions.
* `extensions` list of extensions to create, for example `['vector',
'pg_search']`, `postgis` is created automatically if there's any point
field

### Why?
It's essential to support that field type, especially if the postgres
adapter should be out of beta on 3.0 stable.

### How?
* Bumps `drizzle-orm` to `0.36.1` and `drizzle-kit` to `0.28.0` as we
need this change https://github.com/drizzle-team/drizzle-orm/pull/3141
* Uses its functions to achieve querying functionality, for example the
`near` operator works through `ST_DWithin` or `intersects` through
`ST_Intersects`.
* Removes MongoDB condition from all point field tests, but keeps for
SQLite

Resolves these discussions:
https://github.com/payloadcms/payload/discussions/8996
https://github.com/payloadcms/payload/discussions/8644
2024-11-11 09:31:47 -05:00
Dan Ribbens
ee117bb616 fix!: handle custom id logic in mongodb adapter (#9069)
### What?
Moved the logic for copying the data.id to data._id to the mongoose
adapter.

### Why?
If you have any hooks that need to set the `id`, the value does not get
sent to mongodb as you would expect since it was copied before the
beforeValidate hooks.

### How?
Now data._id is assigned only in the mongodb adapter's `create`
function.

BREAKING CHANGES:
When using custom ID fields, if you have any collection hooks for
beforeValidate, beforeChange then `data._id` will no longer be assigned
as this happens now in the database adapter. Use `data.id` instead.
2024-11-08 15:34:19 -05:00
Sasha
f67761fe22 fix(db-mongodb): totalDocs with joins (#9056)
### What?
Fixes issue with incorrect `totalDocs` value when an aggregation is used
for `find`.
Previously, `limit: 5` for example always returned `totalDocs: 5`.

### Why?
`totalDocs` must be returned correctly.

### How?
Removes `$limit` from the pipeline, as `Model.aggregatePaginate` handles
it by itself.
2024-11-08 14:04:24 -05:00
Dan Ribbens
d20445b6f3 fix(db-mongodb): write migrations index file (#9071)
fix: remove 'undefined' written into mongodb migrations
fix: write migrations index file
2024-11-08 13:26:45 -05:00
Elliot DeNolf
f878e35cc7 chore(release): v3.0.0-beta.126 [skip ci] 2024-11-06 16:23:57 -05:00
Elliot DeNolf
8a5f6f044d chore(release): v3.0.0-beta.125 [skip ci] 2024-11-06 10:24:31 -05:00
Dan Ribbens
93a55d1075 feat: add join field config where property (#8973)
### What?

Makes it possible to filter join documents using a `where` added
directly in the config.


### Why?

It makes the join field more powerful for adding contextual meaning to
the documents being returned. For example, maybe you have a
`requiresAction` field that you set and you can have a join that
automatically filters the documents to those that need attention.

### How?

In the database adapter, we merge the requested `where` to the `where`
defined on the field.
On the frontend the results are filtered using the `filterOptions`
property in the component.

Fixes
https://github.com/payloadcms/payload/discussions/8936
https://github.com/payloadcms/payload/discussions/8937

---------

Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
2024-11-06 10:06:25 -05:00
Elliot DeNolf
e390835711 chore(release): v3.0.0-beta.124 [skip ci] 2024-11-04 14:47:38 -05:00
Elliot DeNolf
c33791d1f8 chore(release): v3.0.0-beta.123 [skip ci] 2024-10-31 16:10:52 -04:00
Sasha
3175541c80 fix: select with unnamed tabs (#8966)
### What?
Fixes `select` handling for properties inside of unnamed tabs using the
mongodb adapter.
Additionally, refactors `traverseFields` in drizzle to reuse logic from
groups / collapsible or rows if unnamed.

### Why?
`select` must work for any fields.

### How?
Fixes the `'tab'` case in `buildProjectionFromSelect` to handle when the
field is an unnamed tab.
Adds extra tests for named tabs / unnamed.
2024-10-31 12:06:05 -04:00
Elliot DeNolf
d192f1414d chore(release): v3.0.0-beta.122 [skip ci] 2024-10-30 21:02:15 -04:00
Elliot DeNolf
d89db00295 chore(release): v3.0.0-beta.121 [skip ci] 2024-10-30 14:25:34 -04:00
Sasha
f4041ce6e2 fix(db-mongodb): joins with singular collection name (#8933)
### What?
Properly specifies `$lookup.from` when the collection name is singular.

### Why?
MongoDB can pluralize the collection name and so can be different for
singular ones.

### How?
Uses the collection name from the driver directly
`adapter.collections[slug].collection.name` instead of just `slug`.
2024-10-30 12:06:03 -04:00
Sasha
dae832c288 feat: select fields (#8550)
Adds `select` which is used to specify the field projection for local
and rest API calls. This is available as an optimization to reduce the
payload's of requests and make the database queries more efficient.

Includes:
- [x] generate types for the `select` property
- [x] infer the return type by `select` with 2 modes - include (`field:
true`) and exclude (`field: false`)
- [x] lots of integration tests, including deep fields / localization
etc
- [x] implement the property in db adapters
- [x] implement the property in the local api for most operations
- [x] implement the property in the rest api 
- [x] docs

---------

Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2024-10-29 21:47:18 +00:00
Elliot DeNolf
43fcccab93 chore(release): v3.0.0-beta.120 [skip ci] 2024-10-28 22:08:50 -04:00
Sasha
7a7a2f3918 fix(db-mongodb): query 'in' with a comma-delimited value (#8910)
### What?
Fixes the issue with `in` querying when the collection has a join field.

### Why?
When using `.aggregate`, MongoDB doesn't cast a comma delimited value
for the `$in` operator to an array automatically as it's not handled by
Mongoose.

### How?
Sanitizes the incoming value to an array if it should.

Fixes https://github.com/payloadcms/payload/issues/8901
2024-10-28 19:41:28 -04:00