95ec57575d
### What? Within collections using the `storage-s3` plugins, we eventually start receiving the following warnings: `@smithy/node-http-handler:WARN socket usage at capacity=50 and 156 additional requests are enqueued. See https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/node-configuring-maxsockets.html or increase socketAcquisitionWarningTimeout=(millis) in the NodeHttpHandler config.` Also referenced in this issue: #6382 The [solution](https://github.com/payloadcms/payload/issues/6382#issuecomment-2325468104) provided by @denolfe in that issue only delayed the reappearance of the problem somewhat, but did not resolve it. ### Why? As far as I understand, in the `staticHandler` of the plugin, when getting items from storage, and they are currently cached, the cached results are immediately returned without handling the stream. As per [this](https://github.com/aws/aws-sdk-js-v3/blob/main/supplemental-docs/CLIENTS.md#nodejs-requesthandler) entry in the aws-sdk docs, if the streaming response is not read, or manually destroyed, a socket might not properly close. ### How? Before returning the cached items, manually destroy the streaming response to make certain the socket is being properly closed. Additionally, add an error check to also consume/destroy the streaming response in case an error occurs, to not leave orphaned sockets. Fixes #6382
S3 Storage for Payload
This package provides a simple way to use S3 with Payload.
NOTE: This package removes the need to use @payloadcms/plugin-cloud-storage as was needed in Payload 2.x.
Installation
pnpm add @payloadcms/storage-s3
Usage
- Configure the
collectionsobject to specify which collections should use the AWS S3 adapter. The slug must match one of your existing collection slugs. - The
configobject can be anyS3ClientConfigobject (from@aws-sdk/client-s3). This is highly dependent on your AWS setup. Check the AWS documentation for more information. - When enabled, this package will automatically set
disableLocalStoragetotruefor each collection.
import { s3Storage } from '@payloadcms/storage-s3'
import { Media } from './collections/Media'
import { MediaWithPrefix } from './collections/MediaWithPrefix'
export default buildConfig({
collections: [Media, MediaWithPrefix],
plugins: [
s3Storage({
collections: {
media: true,
'media-with-prefix': {
prefix,
},
},
bucket: process.env.S3_BUCKET,
config: {
credentials: {
accessKeyId: process.env.S3_ACCESS_KEY_ID,
secretAccessKey: process.env.S3_SECRET_ACCESS_KEY,
},
region: process.env.S3_REGION,
// ... Other S3 configuration
},
}),
],
})
Configuration Options
See the the AWS SDK Package and S3ClientConfig object for guidance on AWS S3 configuration.