Setting Up With Object Storage
Object Storage Configuration
Enterprise Analytics employs compute-storage separation architecture that allows for scaling compute and storage independently. As a result, it requires an object store as its persistent storage. The certified supported object storage solutions are:
-
AWS S3
-
NetApp StorageGRID
You should configure an Enterprise Analytics cluster with an S3 bucket. Each node in an Enterprise Analytics cluster must have access to the configured bucket either using an AWS access policy or credentials file. Non-certified storage solutions may also be compatible, but need to be able to perform the S3 operations listed preceding.
Configuring NetApp StorageGRID
Enterprise Analytics works with NetApp StorageGRID v11.9.0.6 or later . Setting Enterprise Analytics with NetApp StorageGRID involves the following steps:
Create a NetApp StorageGRID bucket
Create a bucket in StorageGRID that Enterprise Analytics uses for storing data.
-
Click Create Bucket in the StorageGRID UI.
-
Enter a name for the bucket, for example,
samplebucket
. -
Select region
us-east-1
or the region you want to use. -
Click Continue to create the bucket.
-
Click Create bucket. The bucket is created.
-
In Bucket options tab, select All, this provides the highest guarantee of consistency.
Configure NetApp StorageGRID credentials
Create the Access key ID and Secret access key for the bucket in StorageGRID. Enterprise Analytics uses these credentials to access the bucket.
-
In the Create access key page, Do not set an expiration time is selected by default.
-
Click Create access key to generate the access key.
Configure S3 Access Credentials
You should place the credentials file in the '.aws' directory in the 'Couchbase' user’s home directory.
For example, /home/couchbase/.aws/credentials
& chmod’d
to 600
as a security best practice.
Make sure to enter the credentials under the default profile.
For more details, refer to the AWS credentials.
You can verify that each node was properly configured using AWS CLI:
aws s3 ls s3://bucket/prefix/ --endpoint-url https://storagegrid-endpoint.com
Configure Enterprise Analytics
Use the Couchbase CLI or REST API to configure each Enterprise Analytics node to use the StorageGRID bucket as its object storage. Copy the Access key ID and Secret access key to the credentials file.
-
From the UI, choose "S3-Compatible Storage".
-
Specify the following:
-
StorageGrid Endpoint: for example, http://mystoragegrid.example.com
-
Bucket Name:
samplebucket
-
Bucket Region:
us-east-1
-
Optionally Bucket Path Prefix: for example,
sampleprefix/
-
-
Make sure to check Force Path Style.
-
Click Save & Finish to save the configuration.
Required Object Storage Operations
Other non-certified S3-compatible object storage solutions may be functional, and need to support the following operations:
-
S3-compatible object storage that supports:
-
Strong, global consistency for bucket updates.
-
At least the following S3 operations:
-
GetObject
-
PutObject
-
DeleteObject
-
CopyObject
-
HeadObject
-
ListObjectsV2
-
UploadPart
-
CreateMultipartUpload
-
CompleteMultipartUpload
-
AbortMultipartUpload
-
GetBucketLocation
-
-