Google Cloud Storage

Use your own GCS bucket for Datasaur projects.

File key

A file key is used during project creation to tell Datasaur which file should be used. You can get the file key from the path after the bucket name in the gsutil URI, as shown in the example below.

• Bucket name: datasaur-test

• gsutil URI: gs://datasaur-test/some-folder/image.png

• File key: /some-folder/image.png

Setup

By integrating your bucket with Datasaur, you can create projects using files directly from your GCS bucket.

1. Create Google Cloud Storage bucket

1. Log in to your GCP account, then go to Cloud Storage.

2. Create a new bucket, or select an existing one you want to integrate with Datasaur.

Keep this tab open, we will return to it later.

2. Set up integration in the Settings page

1. Open a new tab, then go to Datasaur and navigate to Settings > External object storage.

2. Click Add external object Storages.

3. Select the GCS option.

4. Enter the bucket name from Step 1.

5. If this is your first time connecting this bucket:

   • You can use the randomly generated value in the form or set your own.

   • Store this value securely for future use.

6. If the bucket was previously connected to another Datasaur workspace:

   • Enter the security token from the existing integration.

   • You can find it under View details in the Settings > External object storage page.

   • If you no longer have access to the previous workspace, use your saved token.

7. Copy the Service Account Email from the form.

3. Set up Google Cloud Storage bucket permissions

This step allows Datasaur to access your bucket using the service account.

1. Return to your GCP Cloud Storage tab.

2. Open the Permissions tab, and click Grant Access.

3. In the New principals field, paste the Service Account Email copied earlier.

4. In the Role field, select Storage Object Admin (under Cloud Storage).

5. Click Save to finish setting up permissions.

4. Set up CORS

1. On the GCP console, click Activate Cloud Shell in the top-right corner. The terminal will open at the bottom of the page.

2. In the cloud shell, run this command to create a JSON file for the CORS configuration:

   echo '[{"origin":["https://app.datasaur.ai"],"method":["GET","PUT","POST"],"responseHeader":["*"]}]' > datasaur-cors-config.json

3. Then, set the CORS configuration to your bucket. Replace <your-bucket-name> with your actual bucket name:

   gsutil cors set datasaur-cors-config.json gs://<your-bucket-name>

4. To verify that the configuration was applied correctly, run:

   gsutil cors get gs://<your-bucket-name>

5. Check connection

Click Check connection to verify that the configuration is correct. If the connection succeeds, you can continue creating the external object storage integration by clicking Add storage.

6. Setup complete

You can now create projects using files stored directly in your GCS bucket. You can also change the default object storage provider at any time from the Settings page.