Google Cloud Storage

Using your own GCS bucket for Datasaur projects

File Key

This attribute will be used when you create a project to tell Datasaur which file should be used. You can get it by using the path after bucket name on gsutil URI. See the example below.

• Bucket name: datasaur-test

• gsutil URI: gs://datasaur-test/some-folder/image.png

• File key: /some-folder/image.png

Setup

By integrating your bucket into Datasaur, you would be able to create projects using your GCS bucket as the source for the project files.

1. Create or Select Google Cloud Storage Bucket

1. Log into your GCP account, then go to Cloud Storage.

2. Create a new bucket or choose the existing one that you want to integrate.

Don't close this tab, we'll get back to this console again.

2. Setup External Object Storage Integration in Datasaur Team Settings

1. Open your team page, then go to Settings > Integrations.

2. Click on "New External Object Storage". A new window will pop up.

3. Choose the GCS option.

4. Input your bucket name from the first step.

5. If this is your first time setting up this bucket in a Datasaur workspace, you can use the random value from the form or change it as needed. It is also recommended to store the value in a secure location.

6. If you have previously connected the bucket to another Datasaur workspace, please enter the security token from that integration. You can find the security token in the View Details menu on the External Object Storage page. If you no longer have access to that workspace, that's when your previous secure location will come in handy.

7. Copy the Service Account Email from the form.

3. Setup Google Cloud Storage Bucket Permission

This step would allow Datasaur to access your bucket using the service account.

1. Go back to the tab console from the first step.

2. Click on Permissions tab, then Grant Access.

3. On the New Principals field, paste in the Service Account Email that you've copied before.

4. On the role field, select Storage Object Admin (under Cloud Storage) predefined role.

5. Click on Save to finish setting up permissions.

4. Setup CORS

1. On the GCP console page, click on Activate Cloud Shell around the top right corner. The shell will be shown on the bottom of your page.

2. In the cloud shell, input this command to create a JSON file for the CORS configuration.

   echo '[{"origin":["https://app.datasaur.ai"],"method":["GET","PUT","POST"],"responseHeader":["*"]}]' > datasaur-cors-config.json

3. Then, set the CORS configuration to your bucket. Replace <your-bucket-name> with your actual bucket name.

   gsutil cors set datasaur-cors-config.json gs://<your-bucket-name>

4. To verify if the configurations have been set right, use this command. Replace <your-bucket-name> with your actual bucket name.

   gsutil cors get gs://<your-bucket-name>

5. Check Connection

Before clicking on the Add storage button, check the connection by clicking on the Check connection button to make sure your setup is done correctly.

If it's a success, you can continue to create the external object storage by clicking Add storage.

6. Good to go!

Now, you will be able to create projects using files directly from your GCS bucket!

If you have any questions or comments, please let us know, and we'll be happy to support you.