Commit Graph

92 Commits

Author SHA1 Message Date
Germán Jabloñski
2a929cf385 chore: fix all lint errors and add mechanisms to prevent them from appearing again (#12401)
I think it's easier to review this PR commit by commit, so I'll explain
it this way:

## Commits
1. [parallelize eslint script (still showing logs results in
serial)](c9ac49c12d):
Previously, `--concurrency 1` was added to the script to make the logs
more readable. However, turborepo has an option specifically for these
use cases: `--log-order=grouped` runs the tasks in parallel but outputs
them serially. As a result, the lint script is now significantly faster.
2. [run pnpm
lint:fix](9c128c276a)
The auto-fix was run, which resolved some eslint errors that were
slipped in due to the use of `no-verify`. Most of these were
`perfectionist` fixes (property ordering) and the removal of unnecessary
assertions. Starting with this PR, this won't happen again in the
future, as we'll be verifying the linter in every PR across the entire
codebase (see commit 7).
3. [fix eslint non-autofixable
errors](700f412a33)
All manual errors have been resolved except for the configuration errors
addressed in commit 5. Most were React compiler violations, which have
been disabled and commented out "TODO" for now. There's also an unused
`use no memo` and a couple of `require` errors.
4. [move react-compiler linter to eslint-config
package](4f7cb4d63a)
To simplify the eslint configuration. My concern was that there would be
a performance regression when used in non-react related packages, but
none was experienced. This is probably because it only runs on .tsx
files.
5. [remove redundant eslint config files and fix
allowDefaultProject](a94347995a)
The main feature introduced by `typescript-eslint` v8 was
`projectService`, which automatically searches each file for the closest
`tsconfig`, greatly simplifying configuration in monorepos
([source](https://typescript-eslint.io/blog/announcing-typescript-eslint-v8#project-service)).
Once I moved `projectService` to `packages/eslint-config`, all the other
configuration files could be easily removed.
I confirmed that pnpm lint still works on individual packages.
The other important change was that the pending eslint errors from
commits 2 and 3 were resolved. That is, some files were giving the
error: "[File] was not found by the project service. Consider either
including it in the tsconfig.json or including it in
allowDefaultProject." Below I copy the explanatory comment I left in the
code:
```ts
// This is necessary because `tsconfig.base.json` defines `"rootDir": "${configDir}/src"`,
// And the following files aren't in src because they aren't transpiled.
// This is typescript-eslint's way of adding files that aren't included in tsconfig.
// See: https://typescript-eslint.io/troubleshooting/typed-linting/#i-get-errors-telling-me--was-not-found-by-the-project-service-consider-either-including-it-in-the-tsconfigjson-or-including-it-in-allowdefaultproject
// The best practice is to have a tsconfig.json that covers ALL files and is used for
// typechecking (with noEmit), and a `tsconfig.build.json` that is used for the build
// (or alternatively, swc, tsup or tsdown). That's what we should ideally do, in which case
// this hardcoded list wouldn't be necessary. Note that these files don't currently go
// through ts, only through eslint.
```

6. [Differentiate errors from warnings in VScode ESLint
Rules](5914d2f48d)
There's no reason to do that. If an eslint rule isn't an error, it
should be disabled or converted to a warning.
7. [Disable skip lint, and lint over the entire repo now that it's
faster](e4b28f1360)
The GitHub action linted only the files that had changed in the PR.
While this seems like a good idea, once exceptions were introduced with
[skip lint], they opened the door to propagating more and more errors.
Often, the linter was skipped, not because someone introduced new
errors, but because they were trying to avoid those that had already
crept in, sometimes accidentally introducing new ones.
On the other hand, `pnpm lint` now runs in parallel (commit 1), so it's
not that slow. Additionally, it runs in parallel with other GitHub
actions like e2e tests, which take much longer, so it can't represent a
bottleneck in CI.
8. [fix lint in next
package](4506595f91)
Small fix missing from commit 5
9. [Merge remote-tracking branch 'origin/main' into
fix-eslint](563d4909c1)
10. [add again eslint.config.js in payload
package](78f6ffcae7)
The comment in the code explains it. Basically, after the merge from
main, the payload package runs out of memory when linting, probably
because it grew in recent PRs. That package will sooner or later
collapse for our tooling, so we may have to split it. It's already too
big.

## Future Actions
- Resolve React compiler violations, as mentioned in commit 3.
- Decouple the `tsconfig` used for typechecking and build across the
entire monorepo (as explained in point 5) to ensure ts coverage even for
files that aren't transpiled (such as scripts).
- Remove the few remaining `eslint.config.js`. I had to leave the
`richtext-lexical` and `next` ones for now. They could be moved to the
root config and scoped to their packages, as we do for example with
`templates/vercel-postgres/**`. However, I couldn't get it to work, I
don't know why.
- Make eslint in the test folder usable. Not only are we not linting
`test` in CI, but now the `pnpm eslint .` command is so large that my
computer freezes. If each suite were its own package, this would be
solved, and dynamic codegen + git hooks to modify tsconfig.base.json
wouldn't be necessary
([related](https://github.com/payloadcms/payload/pull/11984)).
2025-05-19 12:36:40 -03:00
Sasha
219fd01717 fix(db-postgres): allow the same block slug in different places with a different localized value (#12414)
Fixes https://github.com/payloadcms/payload/issues/12409
Now Payload automatically resolves table names conflicts in those cases,
as well as Drizzle relation names.

---------

Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2025-05-15 16:48:41 -04:00
Sasha
1f6efe9a46 fix: respect hidden: true for virtual fields that have reference to a relationship field (#12219)
Previously, `hidden: true` on a virtual field that references a
relationship field didn't work. Now, this field doesn't get calculated
if there's `hidden: true` and no `showHiddenFields` was passed.

Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2025-05-15 16:48:08 -04:00
Sasha
5fce501589 fix(db-postgres): dbName in arrays regression with long generated drizzle relation names (#12237)
Fixes https://github.com/payloadcms/payload/issues/12136 which caused by
regression from https://github.com/payloadcms/payload/pull/11995

The previous PR solved an issue where the generated drizzle relation
name was too long because of Payload field names, for example
```
{
  name: 'thisIsALongFieldNameThatWillCauseAPostgresErrorEvenThoughWeSetAShorterDBName',
  dbName: 'shortname',
  type: 'array',
  fields: [
    {
      name: 'nested_field_1',
      type: 'array',
      dbName: 'short_nested_1',
      fields: [],
    },
    {
      name: 'nested_field_2',
      type: 'text',
    },
  ],
},
```
But it caused regression, when custom `dbName` vice versa caused long
relation names:
```
export const Header: GlobalConfig = {
  slug: 'header',
  fields: [
    {
      name: 'itemsLvl1',
      type: 'array',
      dbName: 'header_items_lvl1',
      fields: [
        {
          name: 'label',
          type: 'text',
        },
        {
          name: 'itemsLvl2',
          type: 'array',
          dbName: 'header_items_lvl2',
          fields: [
            {
              name: 'label',
              type: 'text',
            },
            {
              name: 'itemsLvl3',
              type: 'array',
              dbName: 'header_items_lvl3',
              fields: [
                {
                  name: 'label',
                  type: 'text',
                },
                {
                  name: 'itemsLvl4',
                  type: 'array',
                  dbName: 'header_items_lvl4',
                  fields: [
                    {
                      name: 'label',
                      type: 'text',
                    },
                  ],
                },
              ],
            },
          ],
        },
      ],
    },
  ],
}
```

Notice if you calculate the generated relation name for `itemsLvl4` you
get:

`header__header_items_lvl1__header_items_lvl2__header_items_lvl3_header_items_lvl4`
- 81 characters, Drizzle, for joining shrink the alias to 63 characters
-`header__header_items_lvl1__header_items_lvl2__header_items_lvl3` and
Postgres throws:
```
error: table name "header__header_items_lvl1__header_items_lvl2__header_items_lvl3" specified more than once
```
2025-05-15 13:40:24 -04:00
Sasha
4a56597b92 fix(db-postgres): count crashes when query contains subqueries and doesn't return any rows (#12273)
Fixes https://github.com/payloadcms/payload/issues/12264

Uses safe object access in `countDistinct`, fallbacks to `0`
2025-04-30 16:53:36 +00:00
Sasha
564fdb0e17 fix: virtual relationship fields with select (#12266)
Continuation of https://github.com/payloadcms/payload/pull/12265.

Currently, using `select` on new relationship virtual fields:
```
const doc = await payload.findByID({
  collection: 'virtual-relations',
  depth: 0,
  id,
  select: { postTitle: true },
})
```
doesn't work, because in order to calculate `post.title`, the `post`
field must be selected as well. This PR adds logic that sanitizes the
incoming `select` to include those relationships into `select` (that are
related to selected virtual fields)

---------

Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2025-04-30 12:27:04 -04:00
Sasha
6dc61ae642 fix(db-mongodb): fallback version when not selected (#12158)
When doing `payload.db.queryDrafts` with `select` without `version`, or
simply your select looks like:
`select: { version: { nonExistingField: true } }` - the `queryDrafts`
function will crash because it tries to access the `version` field.
This PR adds a fallback.
2025-04-18 21:43:53 +03:00
Sasha
fdff5871f6 perf: optimize virtual fields that reference ID (#12159)
This PR optimizes the new virtual fields with relationships feature
https://github.com/payloadcms/payload/pull/11805 when the path
references the ID field, for example:
```
{
  name: 'postCategoryID',
  type: 'number',
  virtual: 'post.category.id',
},
```

Previously, we did additional population of `category`, which is
unnecessary as we can always grab the ID from the `category` value
itself. One less querying step.
2025-04-18 21:39:55 +03:00
Sasha
1c99f46e4f feat: queriable / sortable / useAsTitle virtual fields linked with a relationship field (#11805)
This PR adds an ability to specify a virtual field in this way
```js
{
  slug: 'posts',
  fields: [
    {
      name: 'title',
      type: 'text',
      required: true,
    },
  ],
},
{
  slug: 'virtual-relations',
  fields: [
    {
      name: 'postTitle',
      type: 'text',
      virtual: 'post.title',
    },
    {
      name: 'post',
      type: 'relationship',
      relationTo: 'posts',
    },
  ],
},
```

Then, every time you query `virtual-relations`, `postTitle` will be
automatically populated (even if using `depth: 0`) on the db level. This
field also, unlike `virtual: true` is available for querying / sorting /
`useAsTitle`.

Also, the field can be deeply nested to 2 or more relationships, for
example:
```
{
  name: 'postCategoryTitle',
  type: 'text',
  virtual: 'post.category.title',
},
```

Where the current collection has `post` - a relationship to `posts`, the
collection `posts` has `category` that's a relationship to `categories`
and finally `categories` has `title`.
2025-04-16 15:46:18 -04:00
Dan Ribbens
6572bf4ae1 fix(db-sqlite): text field converts to floating point number (#12107)
### What?

Converts numbers passed to a text field to avoid the database/drizzle
from converting it incorrectly.

### Why?

If you have a hook that passes a value to another field you can
experience this problem where drizzle converts a number value for a text
field to a floating point number in sqlite for example.

### How?

Adds logic to `transform/write/traverseFields.ts` to cast text field
values to string.
2025-04-14 17:05:08 -04:00
Sasha
09782be0e0 fix(db-postgres): long array field table aliases cause error even when dbName is used (#11995)
Fixes https://github.com/payloadcms/payload/issues/11975

Previously, this configuration was causing errors in postgres due to
long names, even though `dbName` is used:
```
{
  slug: 'aliases',
  fields: [
    {
      name: 'thisIsALongFieldNameThatWillCauseAPostgresErrorEvenThoughWeSetAShorterDBName',
      dbName: 'shortname',
      type: 'array',
      fields: [
        {
          name: 'nested_field_1',
          type: 'array',
          dbName: 'short_nested_1',
          fields: [],
        },
        {
          name: 'nested_field_2',
          type: 'text',
        },
      ],
    },
  ],
},
```

This is because we were generating Drizzle relation name (for arrays)
always based on the field path and internally, drizzle uses this name
for aliasing. Now, if `dbName` is present, we use `_{dbName}` instead
for the relation name.
2025-04-07 20:12:43 +00:00
Sasha
4ebd3ce668 fix(db-postgres): deleteOne fails when the where query does not resolve to any document (#11632)
Previously, if you called `payload.db.deleteOne` with a `where` query
that does not resolve to anything, an error would be occurred.
2025-04-04 00:46:31 +03:00
Jacob Fletcher
e87521a376 perf(ui): significantly optimize form state component rendering, up to 96% smaller and 75% faster (#11946)
Significantly optimizes the component rendering strategy within the form
state endpoint by precisely rendering only the fields that require it.
This cuts down on server processing and network response sizes when
invoking form state requests **that manipulate array and block rows
which contain server components**, such as rich text fields, custom row
labels, etc. (results listed below).

Here's a breakdown of the issue:

Previously, when manipulating array and block fields, _all_ rows would
render any server components that might exist within them, including
rich text fields. This means that subsequent changes to these fields
would potentially _re-render_ those same components even if they don't
require it.

For example, if you have an array field with a rich text field within
it, adding the first row would cause the rich text field to render,
which is expected. However, when you add a second row, the rich text
field within the first row would render again unnecessarily along with
the new row.

This is especially noticeable for fields with many rows, where every
single row processes its server components and returns RSC data. And
this does not only affect nested rich text fields, but any custom
component defined on the field level, as these are handled in the same
way.

The reason this was necessary in the first place was to ensure that the
server components receive the proper data when they are rendered, such
as the row index and the row's data. Changing one of these rows could
cause the server component to receive the wrong data if it was not
freshly rendered.

While this is still a requirement that rows receive up-to-date props, it
is no longer necessary to render everything.

Here's a breakdown of the actual fix:

This change ensures that only the fields that are actually being
manipulated will be rendered, rather than all rows. The existing rows
will remain in memory on the client, while the newly rendered components
will return from the server. For example, if you add a new row to an
array field, only the new row will render its server components.

To do this, we send the path of the field that is being manipulated to
the server. The server can then use this path to determine for itself
which fields have already been rendered and which ones need required
rendering.

## Results

The following results were gathered by booting up the `form-state` test
suite and seeding 100 array rows, each containing a rich text field. To
invoke a form state request, we navigate to a document within the
"posts" collection, then add a new array row to the list. The result is
then saved to the file system for comparison.

| Test Suite | Collection | Number of Rows | Before | After | Percentage
Change |
|------|------|---------|--------|--------|--------|
| `form-state` | `posts` | 101 | 1.9MB / 266ms | 80KB / 70ms | ~96%
smaller / ~75% faster |

---------

Co-authored-by: James <james@trbl.design>
Co-authored-by: Alessio Gravili <alessio@gravili.de>
2025-04-03 12:27:14 -04:00
Sasha
f310c90211 fix(db-postgres): down migration fails because migrationTableExists doesn't check in the current transaction (#11910)
Fixes https://github.com/payloadcms/payload/issues/11882

Previously, down migration that dropped the `payload_migrations` table
was failing because `migrationTableExists` doesn't check the current
transaction, only in which you can get a `false` value result.
2025-04-03 02:33:34 +03:00
Alessio Gravili
20e975b7c6 feat: sort support for payload.update operation (#11769)
Continuation of https://github.com/payloadcms/payload/pull/11768. This
adds support for `sort` in `payload.update`.

## Example

```ts
const { docs } = await payload.update({
  collection: 'posts',
  data: {
    title: 'updated',
  },
  limit: 5,
  sort: '-numberField', // <= new
  where: {
    id: {
      exists: true,
    },
  },
})
```
2025-03-19 17:22:13 +00:00
Alessio Gravili
e96d3c87e2 feat(db-*): support sort in db.updateMany (#11768)
This adds support for `sort` in `payload.db.updateMany`.

## Example

```ts
const updatedDocs = await payload.db.updateMany({
  collection: 'posts',
  data: {
    title: 'updated',
  },
  limit: 5,
  sort: '-numberField', // <= new
  where: {
    id: {
      exists: true,
    },
  },
})
```
2025-03-19 10:47:58 -06:00
Dan Ribbens
975bbb756f feat: add find to payloadDataLoader to cache local API queries (#11685)
### What?
Extends our dataloader to add a momiozed payload find function. This way
it will cache the query for the same find request using a cacheKey from
find operation args.

### Why?
This was needed internally for `filterOptions` that exist in an array or
other sitautions where you have the same exact query being made and
awaited many times.

### How?

- Added `find` to payloadDataLoader. Marked `@experimental` in case it
needs to change.
- Created a cache key from the args
- Validate filterOptions changed from `payload.find` to
`payloadDataLoader.find`
- Made `payloadDataLoader` no longer optional on `PayloadRequest`, since
other args are required which are created from createLocalReq (context
for example), I don't see a reason why dataLoader shouldn't be required
also.

Example usage: 
```ts
const result = await req.payloadDataLoader.find({
    collection,
    req,
    where,
  })
```
2025-03-18 21:14:33 +00:00
Sasha
f442d22237 feat(db-*): allow to thread id to create operation data without custom IDs (#11709)
Fixes https://github.com/payloadcms/payload/issues/6884

Adds a new flag `acceptIDOnCreate` that allows you to thread your own
`id` to `payload.create` `data`, for example:

```ts
// doc created with id 1
const doc = await payload.create({ collection: 'posts', data: {id: 1, title: "my title"}})
```

```ts
import { Types } from 'mongoose'
const id = new Types.ObjectId().toHexString()
const doc = await payload.create({ collection: 'posts', data: {id, title: "my title"}})
```
2025-03-17 23:48:35 -04:00
Sasha
11d74871ef feat(db-postgres): add vector raw column type (#10422)
Example how you can add a vector column, enable the `vector` extension
and query your embeddings in the included test -

https://github.com/payloadcms/payload/compare/feat/more-types?expand=1#diff-7d876370487cb625eb42ff1ad7cffa78e8327367af3de2930837ed123f5e3ae6R1-R117
2025-03-17 20:50:00 +00:00
Germán Jabloñski
885f580b58 chore: fix typo (rename mognoose to mongoose) (#11653)
Fixes #11652
2025-03-12 23:01:41 +02:00
Sasha
2ad035fb7b feat(db-mongodb): strip keys from the data that don't exist in the schema from read results (#11558)
This change makes so that data that exists in MongoDB but isn't defined
in the Payload config won't be included to `payload.find` /
`payload.db.find` calls. Now we strip all the additional keys.

Consider you have a field named `secretField` that's also `hidden: true`
(or `read: () => false`) that contains some sensitive data. Then you
removed this field from the database and as for now with the MongoDB
adapter this field will be included to the Local API / REST API results
without any consideration, as Payload doesn't know about it anymore.

This also fixes https://github.com/payloadcms/payload/issues/11542 if
you removed / renamed a relationship field from the schema, Payload
won't sanitize ObjectIDs back to strings anymore.

Ideally you should create a migration script that completely removes the
deleted field from the database with `$unset`, but people rarely do
this.

If you still need to keep those fields to the result, this PR allows you
to do this with the new `allowAdditionalKeys: true` flag.
2025-03-06 14:31:38 +00:00
Sasha
bacc0f002a feat: compound indexes (#11512)
### What?
This PR adds ability to define indexes on several fields for collections
(compound indexes).

Example:
```ts
{
  indexes: [{ unique: true, fields: ['title', 'group.name'] }]
}
```

### Why?
This can be used to either speed up querying/sorting by 2 or more fields
at the same time or to ensure uniqueness between several fields.

### How?
Implements this logic in database adapters. Additionally, adds a utility
`getFieldByPath`.
2025-03-05 03:09:24 +02:00
Alessio Gravili
6a3d58bb32 feat(db-*): support limit in db.updateMany (#11488)
This PR adds a new `limit` property to `payload.db.updateMany`. This functionality is required for [migrating our job system to use faster, direct db adapter calls](https://github.com/payloadcms/payload/pull/11489)
2025-03-03 05:32:57 +00:00
Alessio Gravili
41c7413f59 feat(db-*): add updateMany method to database adapter (#11441)
This PR adds a new `payload.db.updateMany` method, which is a more performant way to update multiple documents compared to using `payload.update`.
2025-02-27 20:30:17 -07:00
Sasha
117949b8d9 test: regenerate payload-types.ts for all test suites (#11238)
Regenerates `payload-types.ts` for all test suites.
2025-02-18 00:45:59 +02:00
Jarrod Flesch
d2fe9b0807 fix(db-mongodb): ensures same level operators are respected (#11087)
### What?
If you had multiple operator constraints on a single field, the last one
defined would be the only one used.

Example:
```ts
where: {
  id: {
    in: [doc2.id],
    not_in: [], // <-- only respected this operator constraint
  },
}
```

and
```ts
where: {
  id: {
    not_in: [],
    in: [doc2.id], // <-- only respected this operator constraint
  },
}
```

They would yield different results.

### Why?
The results were not merged into an `$and` query inside parseParams.

### How?
Merges the results within an `$and` constraint.

Fixes https://github.com/payloadcms/payload/issues/10944

Supersedes https://github.com/payloadcms/payload/pull/11011
2025-02-10 16:29:08 -05:00
Sasha
57143b37d0 fix(db-postgres): ensure globals have createdAt, updatedAt and globalType fields (#10938)
Previously, data for globals was inconsistent across database adapters.
In Postgres, globals didn't store correct `createdAt`, `updatedAt`
fields and the `updateGlobal` lacked the `globalType` field. This PR
solves that without introducing schema changes.
2025-02-06 23:48:59 +02:00
Jacob Fletcher
0acaf8a7f7 fix: field paths within hooks (#10638)
Field paths within hooks are not correct.

For example, an unnamed tab containing a group field and nested text
field should have the path:
- `myGroupField.myTextField`

However, within hooks that path is formatted as:
- `_index-1.myGroupField.myTextField`

The leading index shown above should not exist, as this field is
considered top-level since it is located within an unnamed tab.

This discrepancy is only evident through the APIs themselves, such as
when creating a request with invalid data and reading the validation
errors in the response. Form state contains proper field paths, which is
ultimately why this issue was never caught. This is because within the
admin panel we merge the API response with the current form state,
obscuring the underlying issue. This becomes especially obvious in
#10580, where we no longer initialize validation errors within form
state until the form has been submitted, and instead rely solely on the
API response for the initial error state.

Here's comprehensive example of how field paths _should_ be formatted:

```
{
  // ...
  fields: [
    {
      // path: 'topLevelNamedField'
      // schemaPath: 'topLevelNamedField'
      // indexPath: ''
      name: 'topLevelNamedField',
      type: 'text',
    },
    {
      // path: 'array'
      // schemaPath: 'array'
      // indexPath: ''
      name: 'array',
      type: 'array',
      fields: [
        {
          // path: 'array.[n].fieldWithinArray'
          // schemaPath: 'array.fieldWithinArray'
          // indexPath: ''
          name: 'fieldWithinArray',
          type: 'text',
        },
        {
          // path: 'array.[n].nestedArray'
          // schemaPath: 'array.nestedArray'
          // indexPath: ''
          name: 'nestedArray',
          type: 'array',
          fields: [
            {
              // path: 'array.[n].nestedArray.[n].fieldWithinNestedArray'
              // schemaPath: 'array.nestedArray.fieldWithinNestedArray'
              // indexPath: ''
              name: 'fieldWithinNestedArray',
              type: 'text',
            },
          ],
        },
        {
          // path: 'array.[n]._index-2'
          // schemaPath: 'array._index-2'
          // indexPath: '2'
          type: 'row',
          fields: [
            {
              // path: 'array.[n].fieldWithinRowWithinArray'
              // schemaPath: 'array._index-2.fieldWithinRowWithinArray'
              // indexPath: ''
              name: 'fieldWithinRowWithinArray',
              type: 'text',
            },
          ],
        },
      ],
    },
    {
      // path: '_index-2'
      // schemaPath: '_index-2'
      // indexPath: '2'
      type: 'row',
      fields: [
        {
          // path: 'fieldWithinRow'
          // schemaPath: '_index-2.fieldWithinRow'
          // indexPath: ''
          name: 'fieldWithinRow',
          type: 'text',
        },
      ],
    },
    {
      // path: '_index-3'
      // schemaPath: '_index-3'
      // indexPath: '3'
      type: 'tabs',
      tabs: [
        {
          // path: '_index-3-0'
          // schemaPath: '_index-3-0'
          // indexPath: '3-0'
          label: 'Unnamed Tab',
          fields: [
            {
              // path: 'fieldWithinUnnamedTab'
              // schemaPath: '_index-3-0.fieldWithinUnnamedTab'
              // indexPath: ''
              name: 'fieldWithinUnnamedTab',
              type: 'text',
            },
            {
              // path: '_index-3-0-1'
              // schemaPath: '_index-3-0-1'
              // indexPath: '3-0-1'
              type: 'tabs',
              tabs: [
                {
                  // path: '_index-3-0-1-0'
                  // schemaPath: '_index-3-0-1-0'
                  // indexPath: '3-0-1-0'
                  label: 'Nested Unnamed Tab',
                  fields: [
                    {
                      // path: 'fieldWithinNestedUnnamedTab'
                      // schemaPath: '_index-3-0-1-0.fieldWithinNestedUnnamedTab'
                      // indexPath: ''
                      name: 'fieldWithinNestedUnnamedTab',
                      type: 'text',
                    },
                  ],
                },
              ],
            },
          ],
        },
        {
          // path: 'namedTab'
          // schemaPath: '_index-3.namedTab'
          // indexPath: ''
          label: 'Named Tab',
          name: 'namedTab',
          fields: [
            {
              // path: 'namedTab.fieldWithinNamedTab'
              // schemaPath: '_index-3.namedTab.fieldWithinNamedTab'
              // indexPath: ''
              name: 'fieldWithinNamedTab',
              type: 'text',
            },
          ],
        },
      ],
    },
  ]
}
```
2025-01-27 14:41:35 -05:00
Patrik
ad553e967b fix: updates field validation error messages to use labels if applicable (#10601)
### What?

Previously, field error messages displayed in toast notifications used
the field path to reference fields that failed validation. This
path-based approach was necessary to distinguish between fields that
might share the same name when nested inside arrays, groups, rows, or
collapsible fields.

However, the human readability of these paths was lacking, especially
for unnamed fields like rows and collapsible fields. For example:

- A text field inside a row could display as: `_index-0.text`
- A text field nested within multiple arrays could display as:
`items.0.subArray.0.text`

These outputs are technically correct but not user-friendly.

### Why?

While the previous format was helpful for pinpointing the specific field
that caused the validation error, it could be more user-friendly and
clearer to read. The goal is to maintain the same level of accuracy
while improving the readability for both developers and content editors.

### How?

To improve readability, the following changes were made:

1. Use Field Labels Instead of Field Paths:
- The ValidationError component now uses the label prop from the field
config (if available) instead of the field’s name.
       - If a label is provided, it will be used in the error message.
       - If no label exists, it will fall back to the field’s name.

2. Remove _index from Paths for Unnamed Fields (In the validationError
component only):
- For unnamed fields like rows and collapsibles, the _index prefix is
now stripped from the output to make it cleaner.
       - Instead of `_index-0.text`, it now outputs just `Text`.

3. Reformat the Error Path for Readability:
- The error message format has been improved to be more human-readable,
showing the field hierarchy in a structured way with array indices
converted to 1-based numbers.

#### Example transformation:

##### Before:
The following fields are invalid: `items.0.subArray.0.text`

##### After:
The following fields are invalid: `Items 1 > SubArray 1 > Text`
2025-01-17 09:42:46 -05:00
Jacob Fletcher
86ff0a434c test: field level validation errors (#10614)
Continuation of #10575. Field level validations error were incorrectly
throwing uniqueness errors. This was fixed but lacking tests.
2025-01-16 15:55:34 -05:00
Sasha
98666eb016 perf(db-postgres): do not push database schema if not changed (#10155)
Based on https://github.com/payloadcms/payload/pull/10154

If the actual database schema is not changed (no new columns, enums,
indexes, tables) - skip calling Drizzle push. This, potentially can
significantly reduce overhead on reloads in development mode especially
when using remote databases.

If for whatever reason you need to preserve the current behavior you can
use `PAYLOAD_FORCE_DRIZZLE_PUSH=true` env flag.
2024-12-27 10:12:01 -05:00
Sasha
ed0d3395c7 test: ensure data strictness (#10123)
Ensures we don't save and read additional properties to the database
with both, Local API and `payload.db`.
2024-12-22 09:37:32 +02:00
Jacob Fletcher
957867f6e2 fix: ensures generated IDs persist on create (#10089)
IDs that are supplied directly through the API, such as client-side
generated IDs when adding new blocks and array rows, are overwritten on
create. This is because when adding blocks or array rows on the client,
their IDs are generated first before being sent to the server for
processing. Then when the server receives this data, it incorrectly
overrides them to ensure they are unique when using relational DBs. But
this only needs to happen when no ID was supplied on create, or
specifically when duplicating documents via the `beforeDuplicate` hook.
2024-12-20 15:14:23 -05:00
Sasha
4e953530df feat(db-sqlite): add autoIncrement option (#9427)
### What?
Exposes ability to enable
[AUTOINCREMENT](https://www.sqlite.org/autoinc.html) for Primary Keys
which ensures that the same ID cannot be reused from previously deleted
rows.

```ts
sqliteAdapter({
  autoIncrement: true
})
```

### Why?
This may be essential for some systems. Enabled `autoIncrement: true`
also for the SQLite Adapter in our tests, which can be useful when
testing whether the doc was deleted or not when you also have other
create operations.

### How?
Uses Drizzle's `autoIncrement` option.

WARNING:
This cannot be enabled in an existing project without a custom
migration, as it completely changes how primary keys are stored in the
database.
2024-12-20 20:13:28 +00:00
Sasha
07be617963 fix(db-postgres): relationships v2-v3 migration errors when migrating from v2 to stable v3 (#10080)
When migrating from stable v2 to v3, the provided relationships
migration fails because of not having `payload_locked_documents` and
`payload_locked_documents_rels` tables in the migration generated in v2.


![image](https://github.com/user-attachments/assets/0dc57e6a-5e6e-4b74-bcab-70e660f4e939)
2024-12-19 19:09:07 +00:00
Sasha
23f1ed4a48 feat(db-postgres, db-sqlite): drizzle schema generation (#9953)
This PR allows to have full type safety on `payload.drizzle` with a
single command
```sh
pnpm payload generate:db-schema
```
Which generates TypeScript code with Drizzle declarations based on the
current database schema.

Example of generated file with the website template: 
https://gist.github.com/r1tsuu/b8687f211b51d9a3a7e78ba41e8fbf03

Video that shows the power:


https://github.com/user-attachments/assets/3ced958b-ec1d-49f5-9f51-d859d5fae236


We also now proxy drizzle package the same way we do for Lexical so you
don't have to install it (and you shouldn't because you may have version
mismatch).
Instead, you can import from Drizzle like this:
```ts
import {
  pgTable,
  index,
  foreignKey,
  integer,
  text,
  varchar,
  jsonb,
  boolean,
  numeric,
  serial,
  timestamp,
  uniqueIndex,
  pgEnum,
} from '@payloadcms/db-postgres/drizzle/pg-core'
import { sql } from '@payloadcms/db-postgres/drizzle'
import { relations } from '@payloadcms/db-postgres/drizzle/relations'
```


Fixes https://github.com/payloadcms/payload/discussions/4318

In the future we can also support types generation for mongoose / raw
mongodb results.
2024-12-19 11:08:17 -05:00
Sasha
03ff77544e feat(db-sqlite): add idType: 'uuid' support (#10016)
Adds `idType: 'uuid'` to the SQLite adapter support:
```ts
sqliteAdapter({
  idType: 'uuid',
})
```

Achieved through Drizzle's `$defaultFn()`
https://orm.drizzle.team/docs/latest-releases/drizzle-orm-v0283#-added-defaultfn--default-methods-to-column-builders
as SQLite doesn't have native UUID support. Added `sqlite-uuid` to CI.
2024-12-18 22:44:04 -05:00
Sasha
00909ec5c4 fix(db-sqlite): working point field CRUD and default value (#9989)
Previously, the point field with SQLite was incorrectly built to the
schema and not parsed to the result.
Now, it works with SQLite, just doesn't support queries (`near` etc.).
Fixes
https://github.com/payloadcms/payload/pull/9987#discussion_r1885674299
2024-12-16 07:37:47 +02:00
Sasha
5e3963482e fix(db-postgres): payload.db.upsert inserts new rows instead of updating existing ones (#9916)
### What?
`payload.db.updateOne` (and so `payload.db.upsert`) with drizzle
adapters used incoming `where` incorrectly and worked properly only
either if you passed `id` or some where query path required table joins
(like `where: { 'array.title'`) which is also the reason why `upsert`
_worked_ with user preferences specifically, because we need to join the
`preferences_rels` table to query by `user.relationTo` and `user.value`

Fixes https://github.com/payloadcms/payload/issues/9915

This was found here - https://github.com/payloadcms/payload/pull/9913,
the database KV adapter uses `upsert` with `where` by unique fields.
2024-12-12 03:53:57 +02:00
Alessio Gravili
c7138b9aab chore: update generated types for all test suites (#9577) 2024-11-27 20:36:37 +00:00
Sasha
e176b8b764 fix: correct migrations sorting in getMigrations (#9330)
### What?
We sorted migrations by `-name` in `getMigrations` as by assumption from
generated file names, however, it may be not true as the improved (+
unflaked, previously it failed sometimes) test for `migrate:down` can
reproduce. As in result, `migrateDown` / `migrateRefresh` may execute in
order different from `migrate`.

Unflakes the 'should commit multiple operations async'  test.
We shouldn't pass the same `req` that doesn't contain a transaction to
different operations that execute in parallel (via `Promise.all`)
without either creating a transaction before or using
`isolateObjectProperty(req, 'transactionID')`. It leads to a race
condition because operation can commit a wrong transaction, different
from inited
2024-11-25 13:53:44 -05:00
James Mikrut
9b00b59df0 fix: corrects cases of false positive identification of custom id fields (#9245)
This PR fixes cases where you may have a field called `id` within a
group or a named tab, which would have incorrectly been treated as a
custom ID field for the collection.

However, custom IDs need to be defined at the root level - and now
Payload only respects custom IDs defined at the root level.
2024-11-16 16:29:48 +00:00
Dan Ribbens
7c6f41936b feat(db-mongodb)!: update mongoose to 8.8.1 (#9115)
### What?
Upgrades mongoose from 6 to latest `v8.8.1`

Fixes https://github.com/payloadcms/payload/issues/9171

### Why?
Compatibilty with Mongodb Atlas

### How?
- Updates deps
- Changed ObjectId from bson-objectid to use `new Type.ObjectId` from
mongoose for compatibility (only inside of db-mongodb)
- Internal type adjustments

https://github.com/payloadcms/payload/discussions/9088

BREAKING CHANGES:
All projects with existing data having versions enabled, or relationship or upload fields will want to create the predefined migration that converts all strings to ObjectIDs where needed. This can be created using `payload migrate:create --file @payloadcms/mongodb/relationships-v2-v3`.
For projects making use of the exposed Models from mongoose, review the
upgrade guides from [v6 to
v7](https://mongoosejs.com/docs/7.x/docs/migrating_to_7.html) and [v7 to
v8](https://mongoosejs.com/docs/migrating_to_8.html) and make
adjustments as needed.

---------

Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
2024-11-15 12:03:56 -05:00
Sasha
0a15388edb feat(db-postgres): add point field support (#9078)
### What?
Adds full support for the point field to Postgres and Vercel Postgres
adapters through the Postgis extension. Fully the same API as with
MongoDB, including support for `near`, `within` and `intersects`
operators.

Additionally, exposes to adapter args:
*
`tablesFilter`https://orm.drizzle.team/docs/drizzle-kit-push#including-tables-schemas-and-extensions.
* `extensions` list of extensions to create, for example `['vector',
'pg_search']`, `postgis` is created automatically if there's any point
field

### Why?
It's essential to support that field type, especially if the postgres
adapter should be out of beta on 3.0 stable.

### How?
* Bumps `drizzle-orm` to `0.36.1` and `drizzle-kit` to `0.28.0` as we
need this change https://github.com/drizzle-team/drizzle-orm/pull/3141
* Uses its functions to achieve querying functionality, for example the
`near` operator works through `ST_DWithin` or `intersects` through
`ST_Intersects`.
* Removes MongoDB condition from all point field tests, but keeps for
SQLite

Resolves these discussions:
https://github.com/payloadcms/payload/discussions/8996
https://github.com/payloadcms/payload/discussions/8644
2024-11-11 09:31:47 -05:00
Dan Ribbens
e3172f1e39 fix: migrateRefresh migrates without previously ran migrations (#9073)
fix: migrateRefresh migrates without previously ran migrations
chore: adds tests for database migrate:fresh and migrate:refresh

---------

Co-authored-by: Sasha <64744993+r1tsuu@users.noreply.github.com>
2024-11-08 15:35:46 -05:00
Dan Ribbens
ee117bb616 fix!: handle custom id logic in mongodb adapter (#9069)
### What?
Moved the logic for copying the data.id to data._id to the mongoose
adapter.

### Why?
If you have any hooks that need to set the `id`, the value does not get
sent to mongodb as you would expect since it was copied before the
beforeValidate hooks.

### How?
Now data._id is assigned only in the mongodb adapter's `create`
function.

BREAKING CHANGES:
When using custom ID fields, if you have any collection hooks for
beforeValidate, beforeChange then `data._id` will no longer be assigned
as this happens now in the database adapter. Use `data.id` instead.
2024-11-08 15:34:19 -05:00
Dan Ribbens
d20445b6f3 fix(db-mongodb): write migrations index file (#9071)
fix: remove 'undefined' written into mongodb migrations
fix: write migrations index file
2024-11-08 13:26:45 -05:00
Sasha
9056b9fe9b fix(db-mongodb): virtual fields within row / collapsible / tabs (#8733)
Fixes https://github.com/payloadcms/payload/issues/8674
2024-10-17 16:23:45 -04:00
Dan Ribbens
3110c1b01b fix: local update limit (#8704) 2024-10-14 19:05:13 -04:00
Dan Ribbens
d781624a86 feat: add disableTransaction to local api (#8697) 2024-10-14 14:39:20 -04:00