Commit Graph

7 Commits

Author SHA1 Message Date
Sasha
cae300e8e3 perf: flattenedFields collection/global property, remove deep copying in validateQueryPaths (#9299)
### What?
Improves querying performance of the Local API, reduces copying overhead

### How?

Adds `flattenedFields` property for sanitized collection/global config
which contains fields in database schema structure.
For example, It removes rows / collapsible / unnamed tabs and merges
them to the parent `fields`, ignores UI fields, named tabs are added as
`type: 'tab'`.
This simplifies code in places like Drizzle `transform/traverseFields`.
Also, now we can avoid calling `flattenTopLevelFields` which adds some
overhead and use `collection.flattenedFields` / `field.flattenedFields`.

By refactoring `configToJSONSchema.ts` with `flattenedFields` it also
1. Fixes https://github.com/payloadcms/payload/issues/9467
2. Fixes types for UI fields were generated for `select`



Removes this deep copying for each `where` query constraint
58ac784425/packages/payload/src/database/queryValidation/validateQueryPaths.ts (L69-L73)
which potentially can add overhead if you have a large collection/global
config

UPD:
The overhead is even much more than in the benchmark below if you have
Lexical with blocks.

Benchmark in `relationships/int.spec.ts`:
```ts
const now = Date.now()
for (let i = 0; i < 10; i++) {
  const now = Date.now()
  for (let i = 0; i < 300; i++) {
    const query = await payload.find({
      collection: 'chained',
      where: {
        'relation.relation.name': {
          equals: 'third',
        },
        and: [
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
          {
            'relation.relation.name': {
              equals: 'third',
            },
          },
        ],
      },
    })
  }

  payload.logger.info(`#${i + 1} ${Date.now() - now}`)
}
payload.logger.info(`Total ${Date.now() - now}`)
```

Before:
```
[02:11:48] INFO: #1 3682
[02:11:50] INFO: #2 2199
[02:11:54] INFO: #3 3483
[02:11:56] INFO: #4 2516
[02:11:59] INFO: #5 2467
[02:12:01] INFO: #6 1987
[02:12:03] INFO: #7 1986
[02:12:05] INFO: #8 2375
[02:12:07] INFO: #9 2040
[02:12:09] INFO: #10 1920
    [PASS] Relationships > Querying > Nested Querying > should allow querying two levels deep (24667ms)
[02:12:09] INFO: Total 24657
```

After:
```
[02:12:36] INFO: #1 2113
[02:12:38] INFO: #2 1854
[02:12:40] INFO: #3 1700
[02:12:42] INFO: #4 1797
[02:12:44] INFO: #5 2121
[02:12:46] INFO: #6 1774
[02:12:47] INFO: #7 1670
[02:12:49] INFO: #8 1610
[02:12:50] INFO: #9 1596
[02:12:52] INFO: #10 1576
    [PASS] Relationships > Querying > Nested Querying > should allow querying two levels deep (17828ms)
[02:12:52] INFO: Total 17818
```
2024-11-25 10:28:07 -05:00
Sasha
8af00f2deb fix: join field works on collections with versions enabled (#8715)
- Fixes errors with drizzle when building the schema
https://github.com/payloadcms/payload/issues/8680
- Adds `joins` to `db.queryDrafts` to have them when doing `.find` with
`draft: true`
2024-10-22 11:05:55 -04:00
Dan Ribbens
1bf580fac3 feat: join field works with hasMany relationships (#8493)
Join field works on relationships and uploads having `hasMany: true`

---------

Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
2024-10-08 11:40:34 -04:00
Sasha
8acbda078e feat(drizzle): customize schema with before / after init hooks (#8196)
Adds abillity to customize the generated Drizzle schema with
`beforeSchemaInit` and `afterSchemaInit`. Could be useful if you want to
preserve the existing database schema / override the generated one with
features that aren't supported from the Payload config.

## Docs:

### beforeSchemaInit

Runs before the schema is built. You can use this hook to extend your
database structure with tables that won't be managed by Payload.

```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
import { integer, pgTable, serial } from 'drizzle-orm/pg-core'

postgresAdapter({
  beforeSchemaInit: [
    ({ schema, adapter }) => {
      return {
        ...schema,
        tables: {
          ...schema.tables,
          addedTable: pgTable('added_table', {
            id: serial('id').notNull(),
          }),
        },
      }
    },
  ],
})
```

One use case is preserving your existing database structure when
migrating to Payload. By default, Payload drops the current database
schema, which may not be desirable in this scenario.
To quickly generate the Drizzle schema from your database you can use
[Drizzle
Introspection](https://orm.drizzle.team/kit-docs/commands#introspect--pull)
You should get the `schema.ts` file which may look like this:

```ts
import { pgTable, uniqueIndex, serial, varchar, text } from 'drizzle-orm/pg-core'

export const users = pgTable('users', {
  id: serial('id').primaryKey(),
  fullName: text('full_name'),
  phone: varchar('phone', { length: 256 }),
})

export const countries = pgTable(
  'countries',
  {
    id: serial('id').primaryKey(),
    name: varchar('name', { length: 256 }),
  },
  (countries) => {
    return {
      nameIndex: uniqueIndex('name_idx').on(countries.name),
    }
  },
)

```

You can import them into your config and append to the schema with the
`beforeSchemaInit` hook like this:

```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
import { users, countries } from '../drizzle/schema'

postgresAdapter({
  beforeSchemaInit: [
    ({ schema, adapter }) => {
      return {
        ...schema,
        tables: {
          ...schema.tables,
          users,
          countries
        },
      }
    },
  ],
})
```

Make sure Payload doesn't overlap table names with its collections. For
example, if you already have a collection with slug "users", you should
either change the slug or `dbName` to change the table name for this
collection.


### afterSchemaInit

Runs after the Drizzle schema is built. You can use this hook to modify
the schema with features that aren't supported by Payload, or if you
want to add a column that you don't want to be in the Payload config.
To extend a table, Payload exposes `extendTable` utillity to the args.
You can refer to the [Drizzle
documentation](https://orm.drizzle.team/docs/sql-schema-declaration).
The following example adds the `extra_integer_column` column and a
composite index on `country` and `city` columns.

```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
import { index, integer } from 'drizzle-orm/pg-core'
import { buildConfig } from 'payload'

export default buildConfig({
  collections: [
    {
      slug: 'places',
      fields: [
        {
          name: 'country',
          type: 'text',
        },
        {
          name: 'city',
          type: 'text',
        },
      ],
    },
  ],
  db: postgresAdapter({
    afterSchemaInit: [
      ({ schema, extendTable, adapter }) => {
        extendTable({
          table: schema.tables.places,
          columns: {
            extraIntegerColumn: integer('extra_integer_column'),
          },
          extraConfig: (table) => ({
            country_city_composite_index: index('country_city_composite_index').on(
              table.country,
              table.city,
            ),
          }),
        })

        return schema
      },
    ],
  }),
})

```



<!--

For external contributors, please include:

- A summary of the pull request and any related issues it fixes.
- Reasoning for the changes made or any additional context that may be
useful.

Ensure you have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.

 -->
2024-09-25 15:14:03 -04:00
Jessica Chowdhury
b7a0b15786 feat: add publish specific locale (#7669)
## Description

1. Adds ability to publish a specific individual locale (collections and
globals)
2. Shows published locale in versions list and version comparison
3. Adds new int tests to `versions` test suite

- [X] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.

## Type of change

- [X] New feature (non-breaking change which adds functionality)
- [ ] This change requires a documentation update

## Checklist:

- [X] I have added tests that prove my fix is effective or that my
feature works
- [X] Existing test suite passes locally with my changes
- [ ] I have made corresponding changes to the documentation

---------

Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2024-09-16 16:15:29 -04:00
Elliot DeNolf
5fc9f76406 feat: filename compound index (#7651)
Allow a compound index to be used for upload collections via a
`filenameCompoundIndex` field. Previously, `filename` was always treated
as unique.

Usage:

```ts
{
  slug: 'upload-field',
   upload: {
     // Slugs to include in compound index
     filenameCompoundIndex: ['filename', 'alt'],
  },
}
```
2024-08-13 13:55:10 -04:00
Dan Ribbens
09ad6e4280 feat(drizzle): abstract shared sql code to new package (#7320)
- Abstract shared sql code to a new drizzle package
- Adds sqlite package, not ready to publish until drizzle patches some
issues
- Add `transactionOptions` to allow customizing or disabling db
transactions
- Adds "experimental" label to the `schemaName` property until drizzle
patches an issue
2024-07-24 12:43:29 -04:00