With MRAPs

Utilize Multi-Region Access Points (MRAPs) to connect to your nearest S3 bucket

With MRAP support, you can use Amazon S3 via a global access point that transparently routes to the nearest region. This lets Datasaur access your S3 objects across regions via a single endpoint.

File Key

This attribute will be used when you create a project to tell Datasaur which file should be used. You can get it by using the path after bucket name on S3 URI. See the example below.

For example, if the S3 URI: s3://datasaur-test/some-folder/image.png

Bucket name: datasaur-test
File key: /some-folder/image.png

Setup

Enable MRAP for your buckets: Configure an MRAPs to include the S3 buckets Datasaur should access:
1. Go to the S3 console.
2. In the sidebar, click on Multi-Region Access Points, then click the Create Multi-Region Access Point.
3. Complete the form by assigning a name and selecting all relevant buckets. Keep in mind that you cannot add or remove connected buckets from MRAP, so select carefully.
4. Click the Create Multi-Region Access Point.
5. Once created, open the MRAP and copy its ARN for later use.
Setup External Object Storage integration in Datasaur Team Settings: This step should be very similar with other approaches, except for the bucket name part.
1. Open your team page, then go to Settings > Integrations.
2. Click on "Add External Object Storage". A new window will pop up. Do not close the pop up because we will use the External ID and it will be generated each time you close the form.
3. Specifically for the bucket name, fill it with the MRAP ARN (from the above step).
We'll get back to this window later. Let's leave it for now.

Create a policy for Datasaur role in AWS: Grant Datasaur access to the MRAPs along with the buckets. Just ignore this step if you have already configured a policy for accessing the bucket.

In the AWS IAM management console, go to Policies, then click on Create Policy.

Choose the JSON tab and paste the following configurations below. Replace placeholders with the actual bucket names and MRAP alias. The write permission will be used to upload the selected files to your bucket whereas the get bucket location will be used to configure the request based on your bucket's region. Note: You must add all buckets connected to the MRAP to the IAM policy, so that the assumed role can access the objects in it.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowBucketMetadata",
      "Effect": "Allow",
      "Action": [
        "s3:GetBucketLocation"
      ],
      "Resource": [
        "arn:aws:s3:::<your-bucket-1>",
        "arn:aws:s3:::<your-bucket-2>" // Add more if you have more
      ]
    },
    {
      "Sid": "AllowMRAPAccess",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:ListBucketVersions",
        "s3:PutObject",
        "s3:PutObjectAcl",
        "s3:GetObject",
        "s3:GetObjectAcl",
        "s3:DeleteObject",
        "s3:DeleteObjectVersion"
      ],
      "Resource": [
        "arn:aws:s3::123456789012:accesspoint/your-mrap-alias.mrap",
        "arn:aws:s3::123456789012:accesspoint/your-mrap-alias.mrap/object/*",
        "arn:aws:s3:::<your-bucket-1>/*",
        "arn:aws:s3:::<your-bucket-2>/*" // Add more if you have more
      ]
    }
  ]
}

Click on Next: Tags. We don't require tags to be added, but you can add tags here if you want.
Click on Next: Review. Input a name for the AWS Policy, a description (optional), and click on Create Policy.

Create a role for Datasaur: After we've created a policy for your S3 bucket, we need to attach it to a role which will be assumed by Datasaur to access your bucket.
1. Back on the IAM management console, go to Roles, then click on Create role.
2. Choose AWS account in the trusted entity type section.
3. Click on Another AWS account for the radio button. Fill the value from the first step by copying the Datasaur AWS Account ID (682361690817).
4. Check the Require external ID, then paste the external ID from the first step just like above. After that, click on Next.
5. In the Add permissions section, pick the policy that we've just created from the previous step. Then, click on Next.
6. Input a name, (optional) a description, and click on Create role.
7. After that, back on the Roles page, click on your newly created role.
8. Copy the Role ARN from the page and paste it in the Datasaur External Object Storage integration form.
Verify connection / health check: Before you create the integration, click on check connection to make sure your setup is done correctly. If it's a success, you can continue to create the external object storage.
Good to go! Now, you will be able to create projects using files directly from your S3 bucket through MRAPs.

Last updated 1 day ago