Commit Graph

125 Commits

Author SHA1 Message Date
Elliot DeNolf
058bd02ebd chore(release): v3.1.1 [skip ci] 2024-11-25 16:42:21 -05:00
Sasha
cae300e8e3 perf: flattenedFields collection/global property, remove deep copying in validateQueryPaths (#9299)
### What?
Improves querying performance of the Local API, reduces copying overhead

### How?

Adds `flattenedFields` property for sanitized collection/global config
which contains fields in database schema structure.
For example, It removes rows / collapsible / unnamed tabs and merges
them to the parent `fields`, ignores UI fields, named tabs are added as
`type: 'tab'`.
This simplifies code in places like Drizzle `transform/traverseFields`.
Also, now we can avoid calling `flattenTopLevelFields` which adds some
overhead and use `collection.flattenedFields` / `field.flattenedFields`.

By refactoring `configToJSONSchema.ts` with `flattenedFields` it also
1. Fixes https://github.com/payloadcms/payload/issues/9467
2. Fixes types for UI fields were generated for `select`



Removes this deep copying for each `where` query constraint
58ac784425/packages/payload/src/database/queryValidation/validateQueryPaths.ts (L69-L73)
which potentially can add overhead if you have a large collection/global
config

UPD:
The overhead is even much more than in the benchmark below if you have
Lexical with blocks.

Benchmark in `relationships/int.spec.ts`:
```ts
const now = Date.now()
for (let i = 0; i < 10; i++) {
  const now = Date.now()
  for (let i = 0; i < 300; i++) {
    const query = await payload.find({
      collection: 'chained',
      where: {
        'relation.relation.name': {
          equals: 'third',
        },
        and: [
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
        ],
      },
    })
  }

  payload.logger.info(`#${i + 1} ${Date.now() - now}`)
}
payload.logger.info(`Total ${Date.now() - now}`)
```

Before:
```
[02:11:48] INFO: #1 3682
[02:11:50] INFO: #2 2199
[02:11:54] INFO: #3 3483
[02:11:56] INFO: #4 2516
[02:11:59] INFO: #5 2467
[02:12:01] INFO: #6 1987
[02:12:03] INFO: #7 1986
[02:12:05] INFO: #8 2375
[02:12:07] INFO: #9 2040
[02:12:09] INFO: #10 1920
    [PASS] Relationships > Querying > Nested Querying > should allow querying two levels deep (24667ms)
[02:12:09] INFO: Total 24657
```

After:
```
[02:12:36] INFO: #1 2113
[02:12:38] INFO: #2 1854
[02:12:40] INFO: #3 1700
[02:12:42] INFO: #4 1797
[02:12:44] INFO: #5 2121
[02:12:46] INFO: #6 1774
[02:12:47] INFO: #7 1670
[02:12:49] INFO: #8 1610
[02:12:50] INFO: #9 1596
[02:12:52] INFO: #10 1576
    [PASS] Relationships > Querying > Nested Querying > should allow querying two levels deep (17828ms)
[02:12:52] INFO: Total 17818
```
2024-11-25 10:28:07 -05:00
Elliot DeNolf
65ac739da9 chore(release): v3.1.0 [skip ci] 2024-11-22 11:54:08 -05:00
Jessica Chowdhury
fb3242df0a feat: add skip and force accept flags to migration commands (#8843)
1. Adds flag `--skip-empty` to `migrate:create` to bypass the empty
migration file prompt.
    - Blank migration file will not be created if this flag is passed.
3. Adds flag `--force-accept-warning` to `migrate:fresh` to bypass the
drop database prompt
2024-11-22 10:37:54 -05:00
Elliot DeNolf
025d917fa0 chore(release): v3.0.2 [skip ci] 2024-11-20 16:06:04 -05:00
Elliot DeNolf
9ea26638e9 chore(release): v3.0.1 [skip ci] 2024-11-19 14:33:41 -05:00
Elliot DeNolf
6407e577d3 chore(release): v3.0.0 [skip ci] 2024-11-18 20:10:16 -05:00
Elliot DeNolf
fade739f77 chore(release): v3.0.0-beta.135 [skip ci] 2024-11-18 19:19:55 -05:00
Elliot DeNolf
cb6ceaec76 chore(release): v3.0.0-beta.134 [skip ci] 2024-11-17 11:43:43 -05:00
Elliot DeNolf
457be31ed2 chore(release): v3.0.0-beta.133 [skip ci] 2024-11-16 13:59:41 -05:00
Elliot DeNolf
ff8e7bb968 chore(release): v3.0.0-beta.132 [skip ci] 2024-11-15 18:13:02 -05:00
Elliot DeNolf
90686fa50a chore(release): v3.0.0-beta.131 [skip ci] 2024-11-15 15:34:50 -05:00
Elliot DeNolf
5ff1bb366c chore: misc cleanup (#9206)
- Proper error logger usage
- Some no-fallthrough warning cleanup
2024-11-14 11:14:08 -05:00
Elliot DeNolf
6baff8a3ba chore(release): v3.0.0-beta.130 [skip ci] 2024-11-13 14:18:00 -05:00
Elliot DeNolf
8c2fc71149 chore(release): v3.0.0-beta.129 [skip ci] 2024-11-12 17:07:25 -05:00
Elliot DeNolf
d3cd9baa9b chore(release): v3.0.0-beta.128 [skip ci] 2024-11-12 15:52:52 -05:00
Alessio Gravili
4f6651433c chore: ensure all packages have consistent licenses and package.json metadata (#9079) 2024-11-12 10:27:36 -05:00
Alessio Gravili
03291472d6 chore: bump all eslint dependencies, run lint and prettier (#9128)
This fixes a peer dependency error in our monorepo, as
eslint-plugin-jsx-a11y finally supports eslint v9.

Additionally, this officially adds TypeScript 5.6 support for
typescript-eslint.
2024-11-12 10:18:22 -05:00
Jacob Fletcher
c96fa613bc feat!: on demand rsc (#8364)
Currently, Payload renders all custom components on initial compile of
the admin panel. This is problematic for two key reasons:
1. Custom components do not receive contextual data, i.e. fields do not
receive their field data, edit views do not receive their document data,
etc.
2. Components are unnecessarily rendered before they are used

This was initially required to support React Server Components within
the Payload Admin Panel for two key reasons:
1. Fields can be dynamically rendered within arrays, blocks, etc.
2. Documents can be recursively rendered within a "drawer" UI, i.e.
relationship fields
3. Payload supports server/client component composition 

In order to achieve this, components need to be rendered on the server
and passed as "slots" to the client. Currently, the pattern for this is
to render custom server components in the "client config". Then when a
view or field is needed to be rendered, we first check the client config
for a "pre-rendered" component, otherwise render our client-side
fallback component.

But for the reasons listed above, this pattern doesn't exactly make
custom server components very useful within the Payload Admin Panel,
which is where this PR comes in. Now, instead of pre-rendering all
components on initial compile, we're able to render custom components
_on demand_, only as they are needed.

To achieve this, we've established [this
pattern](https://github.com/payloadcms/payload/pull/8481) of React
Server Functions in the Payload Admin Panel. With Server Functions, we
can iterate the Payload Config and return JSX through React's
`text/x-component` content-type. This means we're able to pass
contextual props to custom components, such as data for fields and
views.

## Breaking Changes

1. Add the following to your root layout file, typically located at
`(app)/(payload)/layout.tsx`:

    ```diff
    /* THIS FILE WAS GENERATED AUTOMATICALLY BY PAYLOAD. */
    /* DO NOT MODIFY IT BECAUSE IT COULD BE REWRITTEN AT ANY TIME. */
    + import type { ServerFunctionClient } from 'payload'

    import config from '@payload-config'
    import { RootLayout } from '@payloadcms/next/layouts'
    import { handleServerFunctions } from '@payloadcms/next/utilities'
    import React from 'react'

    import { importMap } from './admin/importMap.js'
    import './custom.scss'

    type Args = {
      children: React.ReactNode
    }

+ const serverFunctions: ServerFunctionClient = async function (args) {
    +  'use server'
    +  return handleServerFunctions({
    +    ...args,
    +    config,
    +    importMap,
    +  })
    + }

    const Layout = ({ children }: Args) => (
      <RootLayout
        config={config}
        importMap={importMap}
    +  serverFunctions={serverFunctions}
      >
        {children}
      </RootLayout>
    )

    export default Layout
    ```

2. If you were previously posting to the `/api/form-state` endpoint, it
no longer exists. Instead, you'll need to invoke the `form-state` Server
Function, which can be done through the _new_ `getFormState` utility:

    ```diff
    - import { getFormState } from '@payloadcms/ui'
    - const { state } = await getFormState({
    -   apiRoute: '',
    -   body: {
    -     // ...
    -   },
    -   serverURL: ''
    - })

    + const { getFormState } = useServerFunctions()
    +
    + const { state } = await getFormState({
    +   // ...
    + })
    ```

## Breaking Changes

```diff
- useFieldProps()
- useCellProps()
```

More details coming soon.

---------

Co-authored-by: Alessio Gravili <alessio@gravili.de>
Co-authored-by: Jarrod Flesch <jarrodmflesch@gmail.com>
Co-authored-by: James <james@trbl.design>
2024-11-11 13:59:05 -05:00
Elliot DeNolf
0c19afcf91 chore(release): v3.0.0-beta.127 [skip ci] 2024-11-11 09:56:14 -05:00
Sasha
0a15388edb feat(db-postgres): add point field support (#9078)
### What?
Adds full support for the point field to Postgres and Vercel Postgres
adapters through the Postgis extension. Fully the same API as with
MongoDB, including support for `near`, `within` and `intersects`
operators.

Additionally, exposes to adapter args:
*
`tablesFilter`https://orm.drizzle.team/docs/drizzle-kit-push#including-tables-schemas-and-extensions.
* `extensions` list of extensions to create, for example `['vector',
'pg_search']`, `postgis` is created automatically if there's any point
field

### Why?
It's essential to support that field type, especially if the postgres
adapter should be out of beta on 3.0 stable.

### How?
* Bumps `drizzle-orm` to `0.36.1` and `drizzle-kit` to `0.28.0` as we
need this change https://github.com/drizzle-team/drizzle-orm/pull/3141
* Uses its functions to achieve querying functionality, for example the
`near` operator works through `ST_DWithin` or `intersects` through
`ST_Intersects`.
* Removes MongoDB condition from all point field tests, but keeps for
SQLite

Resolves these discussions:
https://github.com/payloadcms/payload/discussions/8996
https://github.com/payloadcms/payload/discussions/8644
2024-11-11 09:31:47 -05:00
Alessio Gravili
0cd83f0591 perf: upgrade json-schema-to-typescript and various other dependencies (#9076)
This further reduces the amount of dependencies installed by payload.
Special thanks to @benmccann for this PR
(https://github.com/bcherny/json-schema-to-typescript/pull/639) !

json-schema-to-typescript before this PR (15.0.1):

![CleanShot 2024-11-07 at 15 35
53@2x](https://github.com/user-attachments/assets/83cb671c-82a8-4b59-b488-cf941d673c8e)

14mb, 37 dependencies

json-schema-to-typescript after this PR (15.0.3):

![CleanShot 2024-11-07 at 15 36
08@2x](https://github.com/user-attachments/assets/d9463275-37e5-452e-aca5-9c1bdeb2a435)

11 MB, 14 dependencies

This PR also upgrades:
- console-table-printer
- croner
- get-tsconfig
- jose
- pino-pretty
- ts-essentials
- tsx
2024-11-07 22:54:45 +00:00
Elliot DeNolf
f878e35cc7 chore(release): v3.0.0-beta.126 [skip ci] 2024-11-06 16:23:57 -05:00
Elliot DeNolf
8a5f6f044d chore(release): v3.0.0-beta.125 [skip ci] 2024-11-06 10:24:31 -05:00
Elliot DeNolf
e390835711 chore(release): v3.0.0-beta.124 [skip ci] 2024-11-04 14:47:38 -05:00
Elliot DeNolf
c33791d1f8 chore(release): v3.0.0-beta.123 [skip ci] 2024-10-31 16:10:52 -04:00
Elliot DeNolf
d192f1414d chore(release): v3.0.0-beta.122 [skip ci] 2024-10-30 21:02:15 -04:00
Elliot DeNolf
d89db00295 chore(release): v3.0.0-beta.121 [skip ci] 2024-10-30 14:25:34 -04:00
Elliot DeNolf
43fcccab93 chore(release): v3.0.0-beta.120 [skip ci] 2024-10-28 22:08:50 -04:00
Elliot DeNolf
6c2eecc47e chore(release): v3.0.0-beta.119 [skip ci] 2024-10-25 16:11:53 -04:00
Sasha
9069bd3fac fix(db-postgres): sort by localized fields (#8839)
### What?
Fixes https://github.com/payloadcms/payload/issues/5152 issue related to
sorting by a localized field with SQLite / Postgres database adapters.

### Why?
It was an incorrect behaviour.


### How?
Modifies the `getTableColumnFromPath` file to have correct join
conditions. Previously if you had this structure in the _locales table
_locale title parent
en          A    1
es          B     1
we sorted by everything that's here, but we need to sort only by the
passed locale.

Additionally fixes a typescript error in `dev.ts` that I added here
https://github.com/payloadcms/payload/pull/8834

Also, removes the condition with `joins.length` in `countDistinct`. It
was there as for this issue
https://github.com/payloadcms/payload/issues/4889 because sorting by a
localized property caused duplication. This can simnifically improve
performance for `.find` with nested querying/sorting on large data sets,
because `count(*)` is faster than `count(DISTINCT id)`
2024-10-24 14:47:58 -04:00
Elliot DeNolf
b482da63c6 chore(release): v3.0.0-beta.118 [skip ci] 2024-10-23 22:07:05 -04:00
Sasha
8af00f2deb fix: join field works on collections with versions enabled (#8715)
- Fixes errors with drizzle when building the schema
https://github.com/payloadcms/payload/issues/8680
- Adds `joins` to `db.queryDrafts` to have them when doing `.find` with
`draft: true`
2024-10-22 11:05:55 -04:00
Elliot DeNolf
69125504af chore(release): v3.0.0-beta.117 [skip ci] 2024-10-22 09:33:50 -04:00
Elliot DeNolf
7136515f8d chore(release): v3.0.0-beta.116 [skip ci] 2024-10-17 09:05:45 -04:00
Sasha
90bca15f52 fix(drizzle): enforce uniqueness on index names (#8754)
Fixes https://github.com/payloadcms/payload/issues/8752

Previously, trying to define a config like this:
```ts
{
  type: 'text',
  name: 'someText',
  index: true,
},
{
  type: 'array',
  name: 'some',
  index: true,
  fields: [
    {
      type: 'text',
      name: 'text',
      index: true,
    },
  ],
}
```

Lead to the error:
```
Warning  We've found duplicated index name across public schema. Please rename your index in either the demonstration table or the table with the duplicated index name
```

Now, if we encounter duplicates, we increment the name like this:
`collection_some_text_idx`
`collection_some_text_1_idx`

---------

Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2024-10-17 02:05:27 +00:00
Dan Ribbens
b269d33278 chore: bump drizzle-kit 0.26.2 (#8750) 2024-10-16 16:32:54 -04:00
Elliot DeNolf
0fb92d3a0a chore(release): v3.0.0-beta.115 [skip ci] 2024-10-16 14:20:27 -04:00
Dan Ribbens
7019f22aad chore: bump drizzle-orm 0.35.1 (#8742) 2024-10-16 15:14:20 +00:00
Dan Ribbens
6d0676ab09 chore: bump drizzle-kit 0.26.1 (#8721) 2024-10-15 21:12:40 +00:00
Dan Ribbens
93545f3103 fix(db-postgres, db-sqlite)!: bump drizzle-kit drizzle-orm @libsql/client (#8617)
Inheriting all the fixes from drizzle moving to latest versions

## BREAKING CHANGES
If you have a prior version of @libsql/client installed in your project,
you must upgrade to 0.14.0
2024-10-15 20:20:50 +00:00
Elliot DeNolf
85e87c15fa chore(release): v3.0.0-beta.114 [skip ci] 2024-10-15 09:51:54 -04:00
Sasha
1ffb6c3e13 fix(drizzle): indexes / unique with relationships (#8432)
Fixes https://github.com/payloadcms/payload/issues/8413 and
https://github.com/payloadcms/payload/issues/6460

- Builds indexes for relationships by default in the SQL schema
- Fixes `unique: true` handling with Postgres / SQLite for every type of
relationships (non-polymorphic. hasMany, polymorphic, polymorphic
hasMany) _note_: disables unique for nested to arrays / blocks
relationships in the `_rels` table.
- adds tests

2.0 PR tha ports only indexes creation
https://github.com/payloadcms/payload/pull/8446, because `unique: true`
could be a breaking change if someone has incosistent unique data in the
database.
2024-10-11 13:13:54 -04:00
Elliot DeNolf
067d353cdd chore(release): v3.0.0-beta.113 [skip ci] 2024-10-10 16:42:05 -04:00
Elliot DeNolf
39825dfce5 chore(release): v3.0.0-beta.112 [skip ci] 2024-10-09 09:56:36 -04:00
Dan Ribbens
1bf580fac3 feat: join field works with hasMany relationships (#8493)
Join field works on relationships and uploads having `hasMany: true`

---------

Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
2024-10-08 11:40:34 -04:00
Elliot DeNolf
e4a413eb9a chore(release): v3.0.0-beta.111 [skip ci] 2024-10-04 11:31:06 -07:00
Sasha
0128eedf70 fix(drizzle)!: make radio and select column names to snake_case (#8439)
Fixes https://github.com/payloadcms/payload/issues/8402 and
https://github.com/payloadcms/payload/issues/8027

Before DB column names were camelCase:

![image](https://github.com/user-attachments/assets/d2965bcf-290a-4f86-9bf4-dfe7e8613934)

After this change, they are snake_case:
![Screenshot 2024-10-04
114226](https://github.com/user-attachments/assets/bbc8c20b-6745-4dd3-b0c8-56263a4e37b1)

#### Breaking SQLite / Postgres ⚠️  
If you had any select (not `hasMany: true`) or radio fields with the
name in camelCase, for example:
```ts
{
  name: 'selectReadOnly',
  type: 'select',
  admin: {
    readOnly: true,
  },
  options: [
    {
      label: 'Value One',
      value: 'one',
    },
    {
      label: 'Value Two',
      value: 'two',
    },
  ],
},
```
This previously was mapped to the db column name `"selectReadOnly"`. Now
it's `select_read_only`.
Generate a new migration to rename your columns.
```sh
pnpm payload migrate:create
```
Then select "rename column" for targeted columns and Drizzle will handle
the migration.

---------

Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2024-10-04 16:25:05 +00:00
Dan Ribbens
27b1629927 fix(db-postgres, db-sqlite): joins relation name (#8491)
A join field on a relationship with a camelCase name would cause an
error from drizzle due to the relation name not matching.
2024-09-30 16:47:21 -04:00
Elliot DeNolf
96d99cb361 chore(release): v3.0.0-beta.110 [skip ci] 2024-09-30 13:19:32 -04:00