### What?
Fixes `select` handling for properties inside of unnamed tabs using the
mongodb adapter.
Additionally, refactors `traverseFields` in drizzle to reuse logic from
groups / collapsible or rows if unnamed.
### Why?
`select` must work for any fields.
### How?
Fixes the `'tab'` case in `buildProjectionFromSelect` to handle when the
field is an unnamed tab.
Adds extra tests for named tabs / unnamed.
Adds `select` which is used to specify the field projection for local
and rest API calls. This is available as an optimization to reduce the
payload's of requests and make the database queries more efficient.
Includes:
- [x] generate types for the `select` property
- [x] infer the return type by `select` with 2 modes - include (`field:
true`) and exclude (`field: false`)
- [x] lots of integration tests, including deep fields / localization
etc
- [x] implement the property in db adapters
- [x] implement the property in the local api for most operations
- [x] implement the property in the rest api
- [x] docs
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
### What?
Allow specifying the defaultSort and defaultLimit to use for populating
a join field
### Why?
It is much easier to set defaults rather than be forced to always call
the join query using the query pattern ("?joins[categories][limit]=0").
### How?
See docs and type changes
This change adds support for sort with multiple fields in local API and
REST API. Related discussion #2089
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
### What?
Fixes https://github.com/payloadcms/payload/issues/5152 issue related to
sorting by a localized field with SQLite / Postgres database adapters.
### Why?
It was an incorrect behaviour.
### How?
Modifies the `getTableColumnFromPath` file to have correct join
conditions. Previously if you had this structure in the _locales table
_locale title parent
en A 1
es B 1
we sorted by everything that's here, but we need to sort only by the
passed locale.
Additionally fixes a typescript error in `dev.ts` that I added here
https://github.com/payloadcms/payload/pull/8834
Also, removes the condition with `joins.length` in `countDistinct`. It
was there as for this issue
https://github.com/payloadcms/payload/issues/4889 because sorting by a
localized property caused duplication. This can simnifically improve
performance for `.find` with nested querying/sorting on large data sets,
because `count(*)` is faster than `count(DISTINCT id)`
Fixes https://github.com/payloadcms/payload/issues/8752
Previously, trying to define a config like this:
```ts
{
type: 'text',
name: 'someText',
index: true,
},
{
type: 'array',
name: 'some',
index: true,
fields: [
{
type: 'text',
name: 'text',
index: true,
},
],
}
```
Lead to the error:
```
Warning We've found duplicated index name across public schema. Please rename your index in either the demonstration table or the table with the duplicated index name
```
Now, if we encounter duplicates, we increment the name like this:
`collection_some_text_idx`
`collection_some_text_1_idx`
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
Inheriting all the fixes from drizzle moving to latest versions
## BREAKING CHANGES
If you have a prior version of @libsql/client installed in your project,
you must upgrade to 0.14.0
Filtering by `null` `number` field values or normal values with the
`exists` operator was not working in `postgres` & `sqlite`.
Was previously fixed for `mongodb`
[here](https://github.com/payloadcms/payload/pull/8416)
Now fixed for `postgres` & `sqlite` adapters as well.
Fixes https://github.com/payloadcms/payload/issues/8413 and
https://github.com/payloadcms/payload/issues/6460
- Builds indexes for relationships by default in the SQL schema
- Fixes `unique: true` handling with Postgres / SQLite for every type of
relationships (non-polymorphic. hasMany, polymorphic, polymorphic
hasMany) _note_: disables unique for nested to arrays / blocks
relationships in the `_rels` table.
- adds tests
2.0 PR tha ports only indexes creation
https://github.com/payloadcms/payload/pull/8446, because `unique: true`
could be a breaking change if someone has incosistent unique data in the
database.
Adds `createDatabase` method to Postgres adapters which can be used
either independently like this:
```ts
payload.db.createDatabase({
name: "some-database",
schemaName: "custom-schema"
})
```
Or
```ts
payload.db.createDatabase()
```
Which creates a database from the current configuration, this is used in
`connect` if `autoDatabaseCreate` is set to `true` (default).
You can disable this behaviour with:
```ts
postgresAdapter({ autoDatabaseCreate: false })
```
Example:
<img width="470" alt="image"
src="https://github.com/user-attachments/assets/8d08c79d-9672-454c-af0f-eb802f9dcd99">
This PR makes a more clear gap between `version_createdAt` /
`version_updatedAt` and `createdAt` / `updatedAt` columns / fields in
mongodb.
- `createdAt` - This should be a new value in a new version. Before this
change it was the same all the time. Should remain the same on autosave.
- The same for `updatedAt`, but it should be updated on every change
(including autosave)
- `version_createdAt` - Should remain equal to `createdAt` from the
parent collection / table
- `version_updatedAt` - On a latest version it makes sense this be the
same as `updatedAt` from the parent collection / table, as all the
`version_*` fields should be just synced with it
Fixes https://github.com/payloadcms/payload/issues/8630
- Fixes `hasMany: true` and `localized: true` on the foreign field
- Adds `limit` to the subquery instead of hardcoded `11`.
- Adds the schema path `field.on` to the subquery, without this having 2
or more relationship fields to the same collection breaks joins
- Properly checks if the field is `hasMany`
fixes https://github.com/payloadcms/payload/issues/8455 and
https://github.com/payloadcms/payload/issues/8462
- Builds the `_locale` column for the `_rels` when it's inside of a
localized group / tab
- Properly builds `sanitizedPath` for blocks in the transform-read
function when it's inside of a localized field. This fixes with fields
inside that have its own table (like `_rels`, select `hasMany: true`
etc)
Adds _more_ tests!
Adds abillity to customize the generated Drizzle schema with
`beforeSchemaInit` and `afterSchemaInit`. Could be useful if you want to
preserve the existing database schema / override the generated one with
features that aren't supported from the Payload config.
## Docs:
### beforeSchemaInit
Runs before the schema is built. You can use this hook to extend your
database structure with tables that won't be managed by Payload.
```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
import { integer, pgTable, serial } from 'drizzle-orm/pg-core'
postgresAdapter({
beforeSchemaInit: [
({ schema, adapter }) => {
return {
...schema,
tables: {
...schema.tables,
addedTable: pgTable('added_table', {
id: serial('id').notNull(),
}),
},
}
},
],
})
```
One use case is preserving your existing database structure when
migrating to Payload. By default, Payload drops the current database
schema, which may not be desirable in this scenario.
To quickly generate the Drizzle schema from your database you can use
[Drizzle
Introspection](https://orm.drizzle.team/kit-docs/commands#introspect--pull)
You should get the `schema.ts` file which may look like this:
```ts
import { pgTable, uniqueIndex, serial, varchar, text } from 'drizzle-orm/pg-core'
export const users = pgTable('users', {
id: serial('id').primaryKey(),
fullName: text('full_name'),
phone: varchar('phone', { length: 256 }),
})
export const countries = pgTable(
'countries',
{
id: serial('id').primaryKey(),
name: varchar('name', { length: 256 }),
},
(countries) => {
return {
nameIndex: uniqueIndex('name_idx').on(countries.name),
}
},
)
```
You can import them into your config and append to the schema with the
`beforeSchemaInit` hook like this:
```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
import { users, countries } from '../drizzle/schema'
postgresAdapter({
beforeSchemaInit: [
({ schema, adapter }) => {
return {
...schema,
tables: {
...schema.tables,
users,
countries
},
}
},
],
})
```
Make sure Payload doesn't overlap table names with its collections. For
example, if you already have a collection with slug "users", you should
either change the slug or `dbName` to change the table name for this
collection.
### afterSchemaInit
Runs after the Drizzle schema is built. You can use this hook to modify
the schema with features that aren't supported by Payload, or if you
want to add a column that you don't want to be in the Payload config.
To extend a table, Payload exposes `extendTable` utillity to the args.
You can refer to the [Drizzle
documentation](https://orm.drizzle.team/docs/sql-schema-declaration).
The following example adds the `extra_integer_column` column and a
composite index on `country` and `city` columns.
```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
import { index, integer } from 'drizzle-orm/pg-core'
import { buildConfig } from 'payload'
export default buildConfig({
collections: [
{
slug: 'places',
fields: [
{
name: 'country',
type: 'text',
},
{
name: 'city',
type: 'text',
},
],
},
],
db: postgresAdapter({
afterSchemaInit: [
({ schema, extendTable, adapter }) => {
extendTable({
table: schema.tables.places,
columns: {
extraIntegerColumn: integer('extra_integer_column'),
},
extraConfig: (table) => ({
country_city_composite_index: index('country_city_composite_index').on(
table.country,
table.city,
),
}),
})
return schema
},
],
}),
})
```
<!--
For external contributors, please include:
- A summary of the pull request and any related issues it fixes.
- Reasoning for the changes made or any additional context that may be
useful.
Ensure you have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
-->
This PR addresses these issues with localized groups / tabs with
Postgres / SQLite:
- Array fields inside of localized groups. Fixes
https://github.com/payloadcms/payload/issues/8322
- Select fields with `hasMany: true` inside of localized groups. Related
to 1, but still needed its own additional logic.
- Relationship (non-polymorphic / non has-many) inside of localized
groups. Previously, even just trying to define them in the config led to
a crash. Fixes https://github.com/payloadcms/payload/issues/8308
Ensures test coverage for localized groups.
Previously, this wasn't valid in Postgres / SQLite:
```ts
const res = await payload.find({
collection: 'polymorphic-relationships',
where: {
polymorphic: {
equals: {
relationTo: 'movies',
value: movie.id,
},
},
},
})
```
Now it works and actually in more performant way than this:
```ts
const res = await payload.find({
collection: 'polymorphic-relationships',
where: {
and: [
{
'polymorphic.relationTo': {
equals: 'movies',
},
},
{
'polymorphic.value': {
equals: 'movies',
},
},
],
},
})
```
Why? Because with the object notation, the output SQL is: `movies_id =
1` - checks exactly 1 column in the `*_rels` table, while with the
separate query by `relationTo` and `value` we need to check against
_each_ possible relationship collection with OR.
Fixes https://github.com/payloadcms/payload/issues/7695
Previosuly, trying to append a new item to an array that contains
another array with localized items and enabled versions led to a unique
`_locale` and `_parent_id` error
```ts
{
name: 'nestedArrayLocalized',
type: 'array',
fields: [
{
type: 'array',
name: 'array',
fields: [
{
name: 'text',
type: 'text',
localized: true,
},
],
},
],
}
```
## Description
- Adds a new "join" field type to Payload and is supported by all database adapters
- The UI uses a table view for the new field
- `db-mongodb` changes relationships to be stored as ObjectIDs instead of strings (for now querying works using both types internally to the DB so no data migration should be necessary unless you're querying directly, see breaking changes for details
- Adds a reusable traverseFields utility to Payload to make it easier to work with nested fields, used internally and for plugin maintainers
```ts
export const Categories: CollectionConfig = {
slug: 'categories',
fields: [
{
name: 'relatedPosts',
type: 'join',
collection: 'posts',
on: 'category',
}
]
}
```
BREAKING CHANGES:
All mongodb relationship and upload values will be stored as MongoDB ObjectIDs instead of strings going forward. If you have existing data and you are querying data directly, outside of Payload's APIs, you get different results. For example, a `contains` query will no longer works given a partial ID of a relationship since the ObjectID requires the whole identifier to work.
---------
Co-authored-by: Jacob Fletcher <jacobsfletch@gmail.com>
Co-authored-by: James <james@trbl.design>
Fixes querying using `in` operator by polymorphic relationship value.
The previous PR https://github.com/payloadcms/payload/pull/8191 didn't
handle the case when the incoming query value is an array and therefore
each item of the array can have a different type.
Ensures test coverage
## Description
Adds `virtual` property to the fields config. Providing `true`
completely disables the field in the DB, which is useful for [Virtual
Fields](https://payloadcms.com/blog/learn-how-virtual-fields-can-help-solve-common-cms-challenges)
Disables abillity to query by a field with `virtual: true`.
Currently, they bloat the DB with unused tables / columns, which may as
well introduce additional joins.
Discussion https://github.com/payloadcms/payload/discussions/6270
Prev PR (this one contains only this feature):
https://github.com/payloadcms/payload/pull/6983
- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
## Type of change
<!-- Please delete options that are not relevant. -->
- [x] New feature (non-breaking change which adds functionality)
- [x] This change requires a documentation update
## Checklist:
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
- [x] I have made corresponding changes to the documentation
## Description
1. Adds ability to publish a specific individual locale (collections and
globals)
2. Shows published locale in versions list and version comparison
3. Adds new int tests to `versions` test suite
- [X] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
## Type of change
- [X] New feature (non-breaking change which adds functionality)
- [ ] This change requires a documentation update
## Checklist:
- [X] I have added tests that prove my fix is effective or that my
feature works
- [X] Existing test suite passes locally with my changes
- [ ] I have made corresponding changes to the documentation
---------
Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
Previously, this worked with MongoDB but failed with Postgres / SQLite
when the `slug` field has both `localized: true` and `unique: true`.
```ts
await payload.create({
collection: "posts",
locale: "en",
data: {
slug: "my-post"
}
})
await payload.create({
collection: "posts",
locale: "de",
data: {
slug: "my-post"
}
})
```
Now, we build unique constraints and indexes in combination with the
_locale column. This should also improve query performance for fields
with both index: true and localized: true.
### Migration steps (Postgres/SQLite only)
This change updates the database schema and requires a migration (if you
have any localized fields). To apply it, run the following commands:
```sh
pnpm payload migration:create locale_unique_indexes
pnpm payload migrate
```
Note that if you use `db.push: true` which is a default, you don't have
to run `pnpm payload migrate` in the development mode, only in the
production, as Payload automatically pushes the schema to your DB with
it.
This PR fixes querying by a relationship field that has custom IDs in
`relationTo` with different types.
Now, in this case, we do cast the ID value in the database.
Example of the config / int test that reproduced the issue:
```ts
{
slug: 'posts-a',
fields: [],
},
{
slug: 'posts-b',
fields: [],
},
{
slug: 'posts-custom-id',
fields: [{ name: 'id', type: 'text' }],
},
{
slug: 'roots',
fields: [
{
name: 'rel',
relationTo: ['posts-a', 'posts-b', 'posts-custom-id'],
type: 'relationship',
},
],
},
```
```ts
const postA = await payload.create({ collection: 'posts-a', data: {} })
const postB = await payload.create({ collection: 'posts-b', data: {} })
const postC = await payload.create({
collection: 'posts-custom-id',
data: { id: crypto.randomUUID() },
})
const root_1 = await payload.create({
collection: 'roots',
data: {
rel: {
value: postC.id,
relationTo: 'posts-custom-id',
},
},
})
const res_1 = await payload.find({
collection: 'roots',
where: {
'rel.value': { equals: postC.id },
},
})
// COALESCE types integer and character varying cannot be matched
expect(res_1.totalDocs).toBe(1)
```
<!--
For external contributors, please include:
- A summary of the pull request and any related issues it fixes.
- Reasoning for the changes made or any additional context that may be
useful.
Ensure you have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
-->
Fixes an issue where duplicating documents in Postgres / SQLite would
crash because of a foreign key constraint / unique ID issue when you
have nested arrays / blocks within localized arrays / blocks.
We now run `beforeDuplicate` against all locales prior to
`beforeValidate` and `beforeChange` hooks.
This PR also fixes a series of issues in Postgres / SQLite where you
have localized groups / named tabs, and then arrays / blocks within the
localized groups / named tabs.
Closes https://github.com/payloadcms/payload/issues/7952
Adds support for `in` and `not_in` operator against JSON field filters.
The following queries are now valid in postgres as well, previously it
only worked in mongo
```ts
await payload.find({
collection: 'posts',
where: {
'data.value': {
in: ['12', '13', '14'],
},
},
context: {
disable: true,
},
})
await payload.find({
collection: 'posts',
where: {
'data.value': {
not_in: ['12', '13', '14'],
},
},
context: {
disable: true,
},
})
```
The `BlockField` type is not representative of the underlying "blocks"
field type, which is plural, i.e. `BlocksField`. This is a semantic
change that will better align the type with the field.
## Breaking Changes
Types related to the `blocks` field have change names. If you were using
the `BlockField` or related types in your own applications, simply
change the import name to be plural and instead of singular.
Old (singular):
```ts
import type {
BlockField,
BlockFieldClient,
BlockFieldValidation,
BlockFieldDescriptionClientComponent,
BlockFieldDescriptionServerComponent,
BlockFieldErrorClientComponent,
BlocksFieldErrorServerComponent,
BlockFieldLabelClientComponent,
BlockFieldLabelServerComponent,
} from 'payload'
```
New (plural):
```ts
import type {
BlocksField,
BlocksFieldClient,
BlocksFieldValidation,
BlocksFieldDescriptionClientComponent,
BlocksFieldDescriptionServerComponent,
BlocksFieldErrorClientComponent,
BlocksFieldErrorServerComponent,
BlocksFieldLabelClientComponent,
BlocksFieldLabelServerComponent,
} from 'payload'
```
## Description
Fixes https://github.com/payloadcms/payload/issues/6037
- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
## Type of change
<!-- Please delete options that are not relevant. -->
- [x] Bug fix (non-breaking change which fixes an issue)
## Checklist:
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
## Description
Fixes https://github.com/payloadcms/payload/issues/7109
Example of table structures that lead to the problem with camelCased
group / tab names.
`group_field_array_localized` - `groupField` -> `array` (has a localized
field inside)
`group_field_array_nested_array` - `groupField` -> `array` ->
`nestedArray`
<!-- Please include a summary of the pull request and any related issues
it fixes. Please also include relevant motivation and context. -->
- [x] I have read and understand the
[CONTRIBUTING.md](https://github.com/payloadcms/payload/blob/main/CONTRIBUTING.md)
document in this repository.
## Type of change
<!-- Please delete options that are not relevant. -->
- [x] Bug fix (non-breaking change which fixes an issue)
## Checklist:
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] Existing test suite passes locally with my changes
