This tutorial describes how to use Google Cloud Storage for a filestore managed by the JFrog Artifactory universal repository manager. When the procedure is complete, all artifacts pushed to Artifactory will be stored in Cloud Storage, and all artifacts retrieved through Artifactory will be drawn from Cloud Storage.
Using Cloud Storage for your Artifactory filestore provides a secure, scalable binary repository that is stored remotely and with redundancy, aiding swift data recovery from a local disaster.
To move your Artifactory filestore to Cloud Storage, execute the following steps:
- Ready Google Cloud Storage
- Ready Artifactory
- Configure Artifactory to use Cloud Storage
- Migrate your filestore to Cloud Storage
This tutorial assumes you already have a Google Cloud Platform (GCP) account and have completed the Google Cloud Storage quickstart documentation.
You also need to have a licensed copy of JFrog Artifactory Enterprise Edition. v4.6 or later, and have completed the Artifactory installation procedure.
This tutorial uses billable components of GCP, including:
- Google Cloud Storage
You can use the Pricing Calculator to generate a cost estimate based on your projected production usage.
Acquire the keys necessary for Cloud Storage and Artifactory interoperability, and create a Cloud Storage Bucket to hold your binary repository data.
The API for Google Cloud Storage Interoperability uses HMAC authentication and lets Cloud Storage interoperate with tools written for cloud storage systems.
To use Cloud Storage with Artifactory, turn on the Google Cloud Storage Interoperability API. This API is enabled per project member, not per project. Each member can set a default project and maintain their own access keys.
Once the API is enabled, use the Cloud Storage console to obtain your access key and access secret. You will use these values for the corresponding parameters in Artifactory when you configure it to use Cloud Storage:
| Cloud Storage Value | Description |
|---|---|
| access key | This Cloud Storage account value will be used as the Artifactory parameter identity. |
| access secret | This Cloud Storage account value will be used as the Artifactory parameter credential. |
Use the Cloud Storage Browser to create a storage bucket in Cloud Storage, as the place where artifacts will be stored.
You must prepare Artifactory before assigning Cloud Storage as your filestore. The procedure you use depends on whether it is a new or existing installation of Artifactory.
If you have not yet installed Artifactory Enterprise Edition v4.6 or newer, do so now. Perform a standard installation using the default settings.
If you have an installed version of Artifactory, perform these steps:
-
Backup your current filestore
Just in case something goes wrong, we recommend backing up your current filestore. To preserve your current filestore as a safety measure, perform a complete system backup of Artifactory. -
Make sure your version of Artifactory supports Cloud Storage
If your version of Artifactory is not Enterprise Edition v4.6 or newer, you must perform a standard upgrade using your current settings. -
Stop Artifactory
Shut down Artifactory. It should not be running for the remaining steps.
To use Artifactory's support for Cloud Storage, you must have an enterprise license for your Artifactory installation. Make sure your $ARTIFACTORY_HOME/etc/artifactory.lic file contains your enterprise license key.
Artifactory offers flexible filestore management through the binarystore.xml configuration file located in the $ARTIFACTORY_HOME/etc folder. By modifying this file you can implement a variety of different binary storage configurations.
To configure Artifactory to use a Cloud Storage object storage provider, you need to specify google-storage as the binary provider in the configuration file. You will also need to provde the name of the bucket to use, and the interoperability storage keys.
The following XML snippet shows the default chain that uses google-storage as the binary provider:
<config version="v1">
<chain>
<provider id="cache-fs" type="cache-fs">
<provider id="eventual" type="eventual">
<provider id="retry" type="retry">
<provider id="google-storage" type="google-storage"/>
</provider>
</provider>
</provider>
</chain>
<!-- Here is an example configuration part for the google-storage: -->
<provider id="google-storage" type="google-storage">
<endpoint>commondatastorage.googleapis.com</endpoint>
<bucketName><NAME></bucketName>
<identity>XXXXXX</identity>
<credential>XXXXXXX</credential>
</provider>
</config>You should use the interoperability values from your Cloud Storage account (as described in Step 1) for identity and credential. The name of the bucket you created in Cloud Storage should be used as the bucketName.
The full set of parameters available for the google-storage binary provider offer other options that may be useful.
Artifactory uses the JetS3t framework to access Cloud Storage. Some of the parameters below are used to set the corresponding value in the framework. For more details, please refer to the JetS3t Configuration Guide.
| Parameter | Description |
|---|---|
| testConnection | Default: true When true, the Artifactory uploads and downloads a file when starting up to verify that the connection to the cloud storage provider is fully functional. |
| multiPartLimit | Default: 100,000,000 bytes File size threshold over which file uploads are chunked and multi-threaded. |
| identity | Your cloud storage provider identity. |
| credential | Your cloud storage provider authentication credential. |
| bucketName | Your globally unique bucket name on Cloud Storage. |
| path | The relative path to your files within the bucket |
| proxyIdentity proxyCredential proxyPort proxyHost | Corresponding parameters if you are accessing the cloud storage provider through a proxy server. |
| port | Default: 80 The port number through which you want to access Cloud Storage. You should only use the default value unless you need to contact a different endpoint for testing purposes. |
| endpoint | Default: commondatastorage.googleapis.com. The Cloud Storage hostname. You should only use the default value unless you need to contact a different endpoint for testing purposes. |
| httpsOnly | Default: false. Set to true if you only want to access Cloud Storage through a secure https connection. |
| httpsPort | Default: 443 The port number through which you want to access Cloud Storage securely through https. You should only use the default value unless you need to contact a different endpoint for testing purposes. |
| bucketExists | Default: false. Only available on google-storage. When true, it indicates to the binary provider that a bucket already exists in Google Cloud Storage and therefore does not need to be created. |
For more information, see the Artifactory User's Guide topic Configuring the Filestore.
To migrate your filestore, execute the following steps:
- Copy the
$ARTIFACTORY_HOME/data/filestoredirectory to your Cloud Storage bucket . - Start Artifactory