Commit Graph

19 Commits

Author SHA1 Message Date
Patrik
faed3aaf26 fix: versions created with incomplete data when using select parameter (#13809)
### What?

Remove `select` parameter from database operations in update operation
during version creation to fix malformed version data.

### Why?

The `select` parameter was being passed to `saveVersion()` in update
operations, causing versions to only contain selected fields

### How?

- Removed `select` parameter from `payload.db.updateOne()` calls in
update operations
- Removed `select` parameter from `saveVersion()` call in update
operation


---
- To see the specific tasks where the Asana app for GitHub is being
used, see below:
  - https://app.asana.com/0/0/1211334352350013
2025-09-15 14:00:04 -07:00
Paul
2d91cb613c feat: allow joins, select, populate, depth and draft to /me REST API operation (#13116)
While we can use `joins`, `select`, `populate`, `depth` or `draft` on
auth collections when finding or finding by ID, these arguments weren't
supported for `/me` which meant that in some situations like in our
ecommerce template we couldn't optimise these calls.

A workaround would be to make a call to `/me` and then get the user ID
to then use for a `findByID` operation.
2025-07-10 17:44:05 +00:00
Alessio Gravili
7c05c775cb docs: improve jobs autorun docs, adds e2e test (#12196)
This clarifies that jobs.autoRun only *runs* already-queued jobs. It does not queue the jobs for you.

Also adds an e2e test as this functionality had no e2e coverage
2025-06-05 09:19:19 -07:00
Sasha
5e3d07bf44 feat: add forceSelect collection / global config property (#11627)
### What?
Adds a new property to collection / global config `forceSelect` which
can be used to ensure that some fields are always selected, regardless
of the `select` query.

### Why?
This can be beneficial for hooks and access control, for example imagine
you need the value of `data.slug` in your hook.
With the following query it would be `undefined`:
`?select[title]=true`
Now, to solve this you can specify
```
forceSelect: {
  slug: true
}
```

### How?
Every operation now merges the incoming `select` with
`collectionConfig.forceSelect`.
2025-03-13 22:04:53 +02:00
Sasha
526e535763 fix: ensure custom IDs are returned to the result when select query exists (#11400)
Previously, behavior with custom IDs and `select` query was incorrect.
By default, the `id` field is guaranteed to be selected, even if it
doesn't exist in the `select` query, this wasn't true for custom IDs.
2025-02-26 17:05:50 +02:00
Sasha
1dc748d341 perf(db-mongodb): remove JSON.parse(JSON.stringify) copying of results (#11293)
Improves performance and optimizes memory usage for mongodb adapter by
cutting down copying of results via `JSON.parse(JSON.stringify())`.
Instead, `transform` does necessary transformations (`ObjectID` ->
`string,` `Date` -> `string`) without any copying
2025-02-21 17:31:24 +02:00
Sasha
e78500feef test: resolves select type errors (#11235)
This PR fixes all typescript errors inside the `select` test suite.
2025-02-18 00:33:44 +02:00
Sasha
938472bf1f fix: populate is ignored for nested relationships (#11227)
### What?
As described in https://github.com/payloadcms/payload/issues/11209,
previously, the `populate` argument was ignored for nested
relationships.

### Why?
`populate` should work for nested relationships, no matter where they
are in the tree.

### How?
Preserves the `populate` argument in the payload data loader.

Fixes https://github.com/payloadcms/payload/issues/11209
2025-02-17 19:44:21 +02:00
Sasha
b08ff88fbd fix(db-mongodb): mongodb optimizations (#10120)
There are few issues introduced in #9594 that we need to look into with
these changes.
2024-12-21 07:42:44 -05:00
Sasha
e468292039 perf(db-mongodb): improve performance of all operations, up to 50% faster (#9594)
This PR improves speed and memory efficiency across all operations with
the Mongoose adapter.

### How?

- Removes Mongoose layer from all database calls, instead uses MongoDB
directly. (this doesn't remove building mongoose schema since it's still
needed for indexes + users in theory can use it)
- Replaces deep copying of read results using
`JSON.parse(JSON.stringify(data))` with the `transform` `operation:
'read'` function which converts Date's, ObjectID's in relationships /
joins to strings. As before, it also handles transformations for write
operations.
- Faster `hasNearConstraint` for potentially large `where`'s
- `traverseFields` now can accept `flattenedFields` which we use in
`transform`. Less recursive calls with tabs/rows/collapsible

Additional fixes
- Uses current transaction for querying nested relationships properties
in `buildQuery`, previously it wasn't used which could've led to wrong
results
- Allows to clear not required point fields with passing `null` from the
Local API. Previously it didn't work in both, MongoDB and Postgres

Benchmarks using this file
https://github.com/payloadcms/payload/blob/chore/db-benchmark/test/_community/int.spec.ts

### Small Dataset Performance

| Metric | Before Optimization | After Optimization | Improvement (%) |

|---------------------------|---------------------|--------------------|-----------------|
| Average FULL (ms) | 1170 | 844 | 27.86% |
| `payload.db.create` (ms) | 1413 | 691 | 51.12% |
| `payload.db.find` (ms) | 2856 | 2204 | 22.83% |
| `payload.db.deleteMany` (ms) | 15206 | 8439 | 44.53% |
| `payload.db.updateOne` (ms) | 21444 | 12162 | 43.30% |
| `payload.db.findOne` (ms) | 159 | 112 | 29.56% |
| `payload.db.deleteOne` (ms) | 3729 | 2578 | 30.89% |
| DB small FULL (ms) | 64473 | 46451 | 27.93% |

---

### Medium Dataset Performance

| Metric | Before Optimization | After Optimization | Improvement (%) |

|---------------------------|---------------------|--------------------|-----------------|
| Average FULL (ms) | 9407 | 6210 | 33.99% |
| `payload.db.create` (ms) | 10270 | 4321 | 57.93% |
| `payload.db.find` (ms) | 20814 | 16036 | 22.93% |
| `payload.db.deleteMany` (ms) | 126351 | 61789 | 51.11% |
| `payload.db.updateOne` (ms) | 201782 | 99943 | 50.49% |
| `payload.db.findOne` (ms) | 1081 | 817 | 24.43% |
| `payload.db.deleteOne` (ms) | 28534 | 23363 | 18.12% |
| DB medium FULL (ms) | 519518 | 342194 | 34.13% |

---

### Large Dataset Performance

| Metric | Before Optimization | After Optimization | Improvement (%) |

|---------------------------|---------------------|--------------------|-----------------|
| Average FULL (ms) | 26575 | 17509 | 34.14% |
| `payload.db.create` (ms) | 29085 | 12196 | 58.08% |
| `payload.db.find` (ms) | 58497 | 43838 | 25.04% |
| `payload.db.deleteMany` (ms) | 372195 | 173218 | 53.47% |
| `payload.db.updateOne` (ms) | 544089 | 288350 | 47.00% |
| `payload.db.findOne` (ms) | 3058 | 2197 | 28.14% |
| `payload.db.deleteOne` (ms) | 82444 | 64730 | 21.49% |
| DB large FULL (ms) | 1461097 | 969714 | 33.62% |
2024-12-19 13:20:39 -05:00
Sasha
2ee3e30b50 fix(db-postgres): select query on upload fields with hasMany: true (#10029)
Previously, if you selected only upload `hasMany: true` field, you would
receive an empty arrays always, because the `_rels` table wasn't joined
in this case. Fixes the condition to count `field.type === 'upload'` .
2024-12-18 06:33:12 +02:00
Sasha
7def6b7ddf fix: defaultPopulate and populate with nested to arrays/blocks properties (#9751)
Fixes https://github.com/payloadcms/payload/issues/9718

### What?
`defaultPopulate` and `populate` didn't work properly when defining
nested to arrays and blocks properties:
```ts
import type { CollectionConfig } from 'payload'
export const Pages: CollectionConfig<'pages'> = {
  slug: 'pages',
  defaultPopulate: {
    slug: true,
    array: {
      title: true,
    },
    blocks: {
      some: {
        title: true,
      },
    },
  },
  access: { read: () => true },
  fields: [
    {
      name: 'slug',
      type: 'text',
      required: true,
    },
    {
      name: 'additional',
      type: 'text',
    },
    {
      name: 'array',
      type: 'array',
      fields: [
        {
          name: 'title',
          type: 'text',
        },
        {
          name: 'other',
          type: 'text',
        },
      ],
    },
    {
      name: 'blocks',
      type: 'blocks',
      blocks: [
        {
          slug: 'some',
          fields: [
            {
              name: 'title',
              type: 'text',
            },
            {
              name: 'other',
              type: 'text',
            },
          ],
        },
      ],
    },
  ],
}

``` 

### Why?
This should work

### How?
Turns out, it wasn't a great idea to mutate passed `select` directly in
`afterRead/promise.ts` to force select some properties `id` ,
`blockType`. Now we do shallow copies when needed.
2024-12-05 12:29:22 -05:00
Sasha
dff71eedaa fix(db-postgres): handle select query on select fields (#9607)
Fixes https://github.com/payloadcms/payload/issues/9606

With Postgres / SQLite, select fields (non `hasMany: true`) weren't
properly handled in the `traverseFields.ts` function for `select` query.
2024-12-04 11:27:03 -05:00
Sasha
0a15388edb feat(db-postgres): add point field support (#9078)
### What?
Adds full support for the point field to Postgres and Vercel Postgres
adapters through the Postgis extension. Fully the same API as with
MongoDB, including support for `near`, `within` and `intersects`
operators.

Additionally, exposes to adapter args:
*
`tablesFilter`https://orm.drizzle.team/docs/drizzle-kit-push#including-tables-schemas-and-extensions.
* `extensions` list of extensions to create, for example `['vector',
'pg_search']`, `postgis` is created automatically if there's any point
field

### Why?
It's essential to support that field type, especially if the postgres
adapter should be out of beta on 3.0 stable.

### How?
* Bumps `drizzle-orm` to `0.36.1` and `drizzle-kit` to `0.28.0` as we
need this change https://github.com/drizzle-team/drizzle-orm/pull/3141
* Uses its functions to achieve querying functionality, for example the
`near` operator works through `ST_DWithin` or `intersects` through
`ST_Intersects`.
* Removes MongoDB condition from all point field tests, but keeps for
SQLite

Resolves these discussions:
https://github.com/payloadcms/payload/discussions/8996
https://github.com/payloadcms/payload/discussions/8644
2024-11-11 09:31:47 -05:00
Sasha
721ae79716 fix: populate with find operation (#9087)
### What?
Because of my error, we didn't pass `populate` to `findOperation` from
the Local API.

### Why?
`populate` must work for every operation that has `depth`.

### How?
Passes `populate` in `operations/local/find.ts`, ensures it works with
the test, checked that other operations pass it.
2024-11-08 18:45:11 +02:00
Sasha
a22c0e62fa feat: add populate property to Local / REST API (#8969)
### What?
Adds `populate` property to Local API and REST API operations that can
be used to specify `select` for a specific collection when it's
populated
```ts
const result = await payload.findByID({
  populate: {
   // type safe if you have generated types
    posts: {
      text: true,
    },
  },
  collection: 'pages',
  depth: 1,
  id: aboutPage.id,
})

result.relatedPost // only has text and id properties
``` 

```ts
fetch('https://localhost:3000/api/pages?populate[posts][text]=true') // highlight-line
  .then((res) => res.json())
  .then((data) => console.log(data))
```

It also overrides
[`defaultPopulate`](https://github.com/payloadcms/payload/pull/8934)

Ensures `defaultPopulate` doesn't affect GraphQL.

### How?
Implements the property for all operations that have the `depth`
argument.
2024-11-06 13:50:19 -05:00
Sasha
3175541c80 fix: select with unnamed tabs (#8966)
### What?
Fixes `select` handling for properties inside of unnamed tabs using the
mongodb adapter.
Additionally, refactors `traverseFields` in drizzle to reuse logic from
groups / collapsible or rows if unnamed.

### Why?
`select` must work for any fields.

### How?
Fixes the `'tab'` case in `buildProjectionFromSelect` to handle when the
field is an unnamed tab.
Adds extra tests for named tabs / unnamed.
2024-10-31 12:06:05 -04:00
Sasha
c41ef65a2b feat: add defaultPopulate property to collection config (#8934)
### What?
Adds `defaultPopulate` property to collection config that allows to
specify which fields to select when the collection is populated from
another document.
```ts
import type { CollectionConfig } from 'payload'

// The TSlug generic can be passed to have type safety for `defaultPopulate`.
// If avoided, the `defaultPopulate` type resolves to `SelectType`.
export const Pages: CollectionConfig<'pages'> = {
  slug: 'pages',
  // I need only slug, NOT the WHOLE CONTENT!
  defaultPopulate: {
    slug: true,
  },
  fields: [
    {
      name: 'slug',
      type: 'text',
      required: true,
    },
  ],
}
```

### Why?
This is essential for example in case of links. You don't need the whole
document, which can contain large data but only the `slug`.

### How?
Implements `defaultPopulate` when populating relationships, including
inside of lexical / slate rich text fields.
2024-10-30 13:41:34 -04:00
Sasha
dae832c288 feat: select fields (#8550)
Adds `select` which is used to specify the field projection for local
and rest API calls. This is available as an optimization to reduce the
payload's of requests and make the database queries more efficient.

Includes:
- [x] generate types for the `select` property
- [x] infer the return type by `select` with 2 modes - include (`field:
true`) and exclude (`field: false`)
- [x] lots of integration tests, including deep fields / localization
etc
- [x] implement the property in db adapters
- [x] implement the property in the local api for most operations
- [x] implement the property in the rest api 
- [x] docs

---------

Co-authored-by: Dan Ribbens <dan.ribbens@gmail.com>
2024-10-29 21:47:18 +00:00