Commit Graph

82 Commits

Author SHA1 Message Date
Elliot DeNolf
058bd02ebd chore(release): v3.1.1 [skip ci] 2024-11-25 16:42:21 -05:00
Sasha
cae300e8e3 perf: flattenedFields collection/global property, remove deep copying in validateQueryPaths (#9299)
### What?
Improves querying performance of the Local API, reduces copying overhead

### How?

Adds `flattenedFields` property for sanitized collection/global config
which contains fields in database schema structure.
For example, It removes rows / collapsible / unnamed tabs and merges
them to the parent `fields`, ignores UI fields, named tabs are added as
`type: 'tab'`.
This simplifies code in places like Drizzle `transform/traverseFields`.
Also, now we can avoid calling `flattenTopLevelFields` which adds some
overhead and use `collection.flattenedFields` / `field.flattenedFields`.

By refactoring `configToJSONSchema.ts` with `flattenedFields` it also
1. Fixes https://github.com/payloadcms/payload/issues/9467
2. Fixes types for UI fields were generated for `select`



Removes this deep copying for each `where` query constraint
58ac784425/packages/payload/src/database/queryValidation/validateQueryPaths.ts (L69-L73)
which potentially can add overhead if you have a large collection/global
config

UPD:
The overhead is even much more than in the benchmark below if you have
Lexical with blocks.

Benchmark in `relationships/int.spec.ts`:
```ts
const now = Date.now()
for (let i = 0; i < 10; i++) {
  const now = Date.now()
  for (let i = 0; i < 300; i++) {
    const query = await payload.find({
      collection: 'chained',
      where: {
        'relation.relation.name': {
          equals: 'third',
        },
        and: [
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
        ],
      },
    })
  }

  payload.logger.info(`#${i + 1} ${Date.now() - now}`)
}
payload.logger.info(`Total ${Date.now() - now}`)
```

Before:
```
[02:11:48] INFO: #1 3682
[02:11:50] INFO: #2 2199
[02:11:54] INFO: #3 3483
[02:11:56] INFO: #4 2516
[02:11:59] INFO: #5 2467
[02:12:01] INFO: #6 1987
[02:12:03] INFO: #7 1986
[02:12:05] INFO: #8 2375
[02:12:07] INFO: #9 2040
[02:12:09] INFO: #10 1920
    [PASS] Relationships > Querying > Nested Querying > should allow querying two levels deep (24667ms)
[02:12:09] INFO: Total 24657
```

After:
```
[02:12:36] INFO: #1 2113
[02:12:38] INFO: #2 1854
[02:12:40] INFO: #3 1700
[02:12:42] INFO: #4 1797
[02:12:44] INFO: #5 2121
[02:12:46] INFO: #6 1774
[02:12:47] INFO: #7 1670
[02:12:49] INFO: #8 1610
[02:12:50] INFO: #9 1596
[02:12:52] INFO: #10 1576
    [PASS] Relationships > Querying > Nested Querying > should allow querying two levels deep (17828ms)
[02:12:52] INFO: Total 17818
```
2024-11-25 10:28:07 -05:00
Elliot DeNolf
65ac739da9 chore(release): v3.1.0 [skip ci] 2024-11-22 11:54:08 -05:00
Elliot DeNolf
025d917fa0 chore(release): v3.0.2 [skip ci] 2024-11-20 16:06:04 -05:00
Elliot DeNolf
9ea26638e9 chore(release): v3.0.1 [skip ci] 2024-11-19 14:33:41 -05:00
Elliot DeNolf
6407e577d3 chore(release): v3.0.0 [skip ci] 2024-11-18 20:10:16 -05:00
Elliot DeNolf
fade739f77 chore(release): v3.0.0-beta.135 [skip ci] 2024-11-18 19:19:55 -05:00
Elliot DeNolf
cb6ceaec76 chore(release): v3.0.0-beta.134 [skip ci] 2024-11-17 11:43:43 -05:00
Elliot DeNolf
457be31ed2 chore(release): v3.0.0-beta.133 [skip ci] 2024-11-16 13:59:41 -05:00
Elliot DeNolf
ff8e7bb968 chore(release): v3.0.0-beta.132 [skip ci] 2024-11-15 18:13:02 -05:00
Elliot DeNolf
90686fa50a chore(release): v3.0.0-beta.131 [skip ci] 2024-11-15 15:34:50 -05:00
Elliot DeNolf
5ff1bb366c chore: misc cleanup (#9206)
- Proper error logger usage
- Some no-fallthrough warning cleanup
2024-11-14 11:14:08 -05:00
Elliot DeNolf
6baff8a3ba chore(release): v3.0.0-beta.130 [skip ci] 2024-11-13 14:18:00 -05:00
Sasha
5b9cee67c0 fix(db-postgres): create relationship-v2-v3 migration (#9178)
### What?
This command from here:
https://github.com/payloadcms/payload/pull/6339
```sh
payload migrate:create --file @payloadcms/db-postgres/relationships-v2-v3
```
stopped working after db-postgers and drizzle packages were separated 

### How?
Passes correct `dirname` to `getPredefinedMigration`

Additionally, adds support for `.js` files in `getPredefinedMigration`
2024-11-13 19:02:17 +00:00
Elliot DeNolf
8c2fc71149 chore(release): v3.0.0-beta.129 [skip ci] 2024-11-12 17:07:25 -05:00
Elliot DeNolf
d3cd9baa9b chore(release): v3.0.0-beta.128 [skip ci] 2024-11-12 15:52:52 -05:00
Alessio Gravili
4f6651433c chore: ensure all packages have consistent licenses and package.json metadata (#9079) 2024-11-12 10:27:36 -05:00
Alessio Gravili
03291472d6 chore: bump all eslint dependencies, run lint and prettier (#9128)
This fixes a peer dependency error in our monorepo, as
eslint-plugin-jsx-a11y finally supports eslint v9.

Additionally, this officially adds TypeScript 5.6 support for
typescript-eslint.
2024-11-12 10:18:22 -05:00
Sasha
b878daf27a feat(db-postgres): deep querying on json and rich text fields (#9102)
### What?
Allows to query on JSON / Rich Text fields in Postgres the same way as
in Mongodb with any nesting level.

Example:
Data:
```js
{
  json: {
    array: [
      {
        text: 'some-text', // nested to array + object
        object: {
          text: 'deep-text', // nested to array + 2x object
          array: [10], // number is nested to array + 2x object + array
        },
      },
    ],
  }
}
```
Query:
```ts
payload.find({
  collection: 'json-fields',
  where: {
    and: [
      {
        'json.array.text': {
          equals: 'some-text',
        },
      },
      {
        'json.array.object.text': {
          equals: 'deep-text',
        },
      },
      {
        'json.array.object.array': {
          in: [10, 20],
        },
      },
      {
        'json.array.object.array': {
          exists: true,
        },
      },
      {
        'json.array.object.notexists': {
          exists: false,
        },
      },
    ],
  },
})
```

### How?
Utilizes [the `jsonb_path_exists` postgres
function](https://www.postgresql.org/docs/current/functions-json.html)
2024-11-12 09:26:04 +02:00
Jacob Fletcher
c96fa613bc feat!: on demand rsc (#8364)
Currently, Payload renders all custom components on initial compile of
the admin panel. This is problematic for two key reasons:
1. Custom components do not receive contextual data, i.e. fields do not
receive their field data, edit views do not receive their document data,
etc.
2. Components are unnecessarily rendered before they are used

This was initially required to support React Server Components within
the Payload Admin Panel for two key reasons:
1. Fields can be dynamically rendered within arrays, blocks, etc.
2. Documents can be recursively rendered within a "drawer" UI, i.e.
relationship fields
3. Payload supports server/client component composition 

In order to achieve this, components need to be rendered on the server
and passed as "slots" to the client. Currently, the pattern for this is
to render custom server components in the "client config". Then when a
view or field is needed to be rendered, we first check the client config
for a "pre-rendered" component, otherwise render our client-side
fallback component.

But for the reasons listed above, this pattern doesn't exactly make
custom server components very useful within the Payload Admin Panel,
which is where this PR comes in. Now, instead of pre-rendering all
components on initial compile, we're able to render custom components
_on demand_, only as they are needed.

To achieve this, we've established [this
pattern](https://github.com/payloadcms/payload/pull/8481) of React
Server Functions in the Payload Admin Panel. With Server Functions, we
can iterate the Payload Config and return JSX through React's
`text/x-component` content-type. This means we're able to pass
contextual props to custom components, such as data for fields and
views.

## Breaking Changes

1. Add the following to your root layout file, typically located at
`(app)/(payload)/layout.tsx`:

    ```diff
    /* THIS FILE WAS GENERATED AUTOMATICALLY BY PAYLOAD. */
    /* DO NOT MODIFY IT BECAUSE IT COULD BE REWRITTEN AT ANY TIME. */
    + import type { ServerFunctionClient } from 'payload'

    import config from '@payload-config'
    import { RootLayout } from '@payloadcms/next/layouts'
    import { handleServerFunctions } from '@payloadcms/next/utilities'
    import React from 'react'

    import { importMap } from './admin/importMap.js'
    import './custom.scss'

    type Args = {
      children: React.ReactNode
    }

+ const serverFunctions: ServerFunctionClient = async function (args) {
    +  'use server'
    +  return handleServerFunctions({
    +    ...args,
    +    config,
    +    importMap,
    +  })
    + }

    const Layout = ({ children }: Args) => (
      <RootLayout
        config={config}
        importMap={importMap}
    +  serverFunctions={serverFunctions}
      >
        {children}
      </RootLayout>
    )

    export default Layout
    ```

2. If you were previously posting to the `/api/form-state` endpoint, it
no longer exists. Instead, you'll need to invoke the `form-state` Server
Function, which can be done through the _new_ `getFormState` utility:

    ```diff
    - import { getFormState } from '@payloadcms/ui'
    - const { state } = await getFormState({
    -   apiRoute: '',
    -   body: {
    -     // ...
    -   },
    -   serverURL: ''
    - })

    + const { getFormState } = useServerFunctions()
    +
    + const { state } = await getFormState({
    +   // ...
    + })
    ```

## Breaking Changes

```diff
- useFieldProps()
- useCellProps()
```

More details coming soon.

---------

Co-authored-by: Alessio Gravili <alessio@gravili.de>
Co-authored-by: Jarrod Flesch <jarrodmflesch@gmail.com>
Co-authored-by: James <james@trbl.design>
2024-11-11 13:59:05 -05:00
Elliot DeNolf
0c19afcf91 chore(release): v3.0.0-beta.127 [skip ci] 2024-11-11 09:56:14 -05:00
Sasha
0a15388edb feat(db-postgres): add point field support (#9078)
### What?
Adds full support for the point field to Postgres and Vercel Postgres
adapters through the Postgis extension. Fully the same API as with
MongoDB, including support for `near`, `within` and `intersects`
operators.

Additionally, exposes to adapter args:
*
`tablesFilter`https://orm.drizzle.team/docs/drizzle-kit-push#including-tables-schemas-and-extensions.
* `extensions` list of extensions to create, for example `['vector',
'pg_search']`, `postgis` is created automatically if there's any point
field

### Why?
It's essential to support that field type, especially if the postgres
adapter should be out of beta on 3.0 stable.

### How?
* Bumps `drizzle-orm` to `0.36.1` and `drizzle-kit` to `0.28.0` as we
need this change https://github.com/drizzle-team/drizzle-orm/pull/3141
* Uses its functions to achieve querying functionality, for example the
`near` operator works through `ST_DWithin` or `intersects` through
`ST_Intersects`.
* Removes MongoDB condition from all point field tests, but keeps for
SQLite

Resolves these discussions:
https://github.com/payloadcms/payload/discussions/8996
https://github.com/payloadcms/payload/discussions/8644
2024-11-11 09:31:47 -05:00
Alessio Gravili
0cd83f0591 perf: upgrade json-schema-to-typescript and various other dependencies (#9076)
This further reduces the amount of dependencies installed by payload.
Special thanks to @benmccann for this PR
(https://github.com/bcherny/json-schema-to-typescript/pull/639) !

json-schema-to-typescript before this PR (15.0.1):

![CleanShot 2024-11-07 at 15 35
53@2x](https://github.com/user-attachments/assets/83cb671c-82a8-4b59-b488-cf941d673c8e)

14mb, 37 dependencies

json-schema-to-typescript after this PR (15.0.3):

![CleanShot 2024-11-07 at 15 36
08@2x](https://github.com/user-attachments/assets/d9463275-37e5-452e-aca5-9c1bdeb2a435)

11 MB, 14 dependencies

This PR also upgrades:
- console-table-printer
- croner
- get-tsconfig
- jose
- pino-pretty
- ts-essentials
- tsx
2024-11-07 22:54:45 +00:00
Elliot DeNolf
f878e35cc7 chore(release): v3.0.0-beta.126 [skip ci] 2024-11-06 16:23:57 -05:00
Elliot DeNolf
8a5f6f044d chore(release): v3.0.0-beta.125 [skip ci] 2024-11-06 10:24:31 -05:00
Elliot DeNolf
e390835711 chore(release): v3.0.0-beta.124 [skip ci] 2024-11-04 14:47:38 -05:00
Elliot DeNolf
c33791d1f8 chore(release): v3.0.0-beta.123 [skip ci] 2024-10-31 16:10:52 -04:00
Elliot DeNolf
d192f1414d chore(release): v3.0.0-beta.122 [skip ci] 2024-10-30 21:02:15 -04:00
Elliot DeNolf
d89db00295 chore(release): v3.0.0-beta.121 [skip ci] 2024-10-30 14:25:34 -04:00
Elliot DeNolf
43fcccab93 chore(release): v3.0.0-beta.120 [skip ci] 2024-10-28 22:08:50 -04:00
Elliot DeNolf
6c2eecc47e chore(release): v3.0.0-beta.119 [skip ci] 2024-10-25 16:11:53 -04:00
Elliot DeNolf
b482da63c6 chore(release): v3.0.0-beta.118 [skip ci] 2024-10-23 22:07:05 -04:00
Elliot DeNolf
69125504af chore(release): v3.0.0-beta.117 [skip ci] 2024-10-22 09:33:50 -04:00
Elliot DeNolf
9fb86655a9 fix: drizzle init args (#8818)
Adjust drizzle init for changes in drizzle 0.35.0
https://github.com/drizzle-team/drizzle-orm/releases/tag/0.35.0

The pool/connection should now be passed as the `client` arg when
initializing drizzle.

```ts
this.drizzle = drizzle({
  client: this.poolOptions ? new VercelPool(this.poolOptions) : sql,
  logger,
  schema: this.schema,
})
```

This was causing an issue where running `payload migrate` on Vercel was
causing drizzle to attempt to `127.0.0.1:5432` instead of the specified
environment variable in the adapter 🤔
2024-10-21 21:39:37 -04:00
Elliot DeNolf
7136515f8d chore(release): v3.0.0-beta.116 [skip ci] 2024-10-17 09:05:45 -04:00
Sasha
90bca15f52 fix(drizzle): enforce uniqueness on index names (#8754)
Fixes https://github.com/payloadcms/payload/issues/8752

Previously, trying to define a config like this:
```ts
{
  type: 'text',
  name: 'someText',
  index: true,
},
{
  type: 'array',
  name: 'some',
  index: true,
  fields: [
    {
      type: 'text',
      name: 'text',
      index: true,
    },
  ],
}
```

Lead to the error:
```
Warning  We've found duplicated index name across public schema. Please rename your index in either the demonstration table or the table with the duplicated index name
```

Now, if we encounter duplicates, we increment the name like this:
`collection_some_text_idx`
`collection_some_text_1_idx`

---------

Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2024-10-17 02:05:27 +00:00
Dan Ribbens
b269d33278 chore: bump drizzle-kit 0.26.2 (#8750) 2024-10-16 16:32:54 -04:00
Elliot DeNolf
0fb92d3a0a chore(release): v3.0.0-beta.115 [skip ci] 2024-10-16 14:20:27 -04:00
Dan Ribbens
7019f22aad chore: bump drizzle-orm 0.35.1 (#8742) 2024-10-16 15:14:20 +00:00
Dan Ribbens
6d0676ab09 chore: bump drizzle-kit 0.26.1 (#8721) 2024-10-15 21:12:40 +00:00
Dan Ribbens
93545f3103 fix(db-postgres, db-sqlite)!: bump drizzle-kit drizzle-orm @libsql/client (#8617)
Inheriting all the fixes from drizzle moving to latest versions

## BREAKING CHANGES
If you have a prior version of @libsql/client installed in your project,
you must upgrade to 0.14.0
2024-10-15 20:20:50 +00:00
Elliot DeNolf
85e87c15fa chore(release): v3.0.0-beta.114 [skip ci] 2024-10-15 09:51:54 -04:00
Sasha
6be665f887 chore(eslint): payload-logger proper usage works with template literals (#8660)
Improves this https://github.com/payloadcms/payload/pull/8578 to work
with template literals as well,

This is invalid:
```ts
this.payload.logger.error('Failed to create database', err)
```

But this is valid:
```ts
this.payload.logger.error(`Failed to create database ${dbName}`, err)
```

This PR fixes that

<img width="577" alt="image"
src="https://github.com/user-attachments/assets/b867da53-4f81-4b1b-8423-598e0419946a">
2024-10-13 19:10:47 -04:00
Sasha
256949e331 feat(db-postgres, db-vercel-postgres): createDatabase, auto database create if it does not exist (#8655)
Adds `createDatabase` method to Postgres adapters which can be used
either independently like this:
```ts
payload.db.createDatabase({
  name: "some-database",
  schemaName: "custom-schema"
})
```
Or
```ts
payload.db.createDatabase()
```
Which creates a database from the current configuration, this is used in
`connect` if `autoDatabaseCreate` is set to `true` (default).
You can disable this behaviour with:
```ts
postgresAdapter({ autoDatabaseCreate: false })
```

Example:
<img width="470" alt="image"
src="https://github.com/user-attachments/assets/8d08c79d-9672-454c-af0f-eb802f9dcd99">
2024-10-11 14:17:32 +00:00
Elliot DeNolf
067d353cdd chore(release): v3.0.0-beta.113 [skip ci] 2024-10-10 16:42:05 -04:00
Elliot DeNolf
39825dfce5 chore(release): v3.0.0-beta.112 [skip ci] 2024-10-09 09:56:36 -04:00
Elliot DeNolf
ca779441a3 fix(db-vercel-postgres): add pg dep (#8598) 2024-10-08 10:32:28 -04:00
Elliot DeNolf
e4a413eb9a chore(release): v3.0.0-beta.111 [skip ci] 2024-10-04 11:31:06 -07:00
Elliot DeNolf
96d99cb361 chore(release): v3.0.0-beta.110 [skip ci] 2024-09-30 13:19:32 -04:00
Dan Ribbens
3847428f0a fix(db-*): make db.begintransactions required (#8419)
Changing the transcations functions on the db so that projects using
typescript `strict: true` do not need to type narrow before using it.
2024-09-30 13:12:07 -04:00