Connect an S3 client

Any S3 client works with Object Storage once it has four settings: our endpoint, a region name, path-style addressing and an access key. This page explains each setting, then gives working setups for the AWS CLI, rclone, s3cmd and the AWS SDKs.

• Create an access key and keep its access key ID and secret key at hand.
• Find the region that holds your bucket. The Region column of the Buckets tab gives its code, and the Endpoints card on the Overview tab gives the endpoint of each region the service uses.
• Allow outgoing HTTPS on TCP port 7337 in any firewall between your client and the internet.

Endpoint

Each region has its own endpoint, and a bucket answers only at the endpoint of its region. Another region's endpoint answers NoSuchBucket for it. Keep the port 7337 in the endpoint. Without it, a client connects to port 443, which does not serve Object Storage.

Region

We accept any region name in the signature. A tool that insists on a region works with the region code, such as fra, or with us-east-1.

Path style

The bucket name goes in the path, as in https://s3.fra.coritan.com:7337/u7-assets/photo.jpg. Hostnames made from a bucket name, such as u7-assets.s3.fra.coritan.com, do not exist, so a client set to virtual-hosted style cannot connect.

The examples below use the fra endpoint, the bucket u7-assets, and the placeholders YOUR_ACCESS_KEY_ID and YOUR_SECRET_KEY. Replace them with your own values. Name the bucket in every command: an access key cannot create or delete buckets, so commands such as aws s3 mb are refused.

Keep the key in a named profile, so the AWS CLI finds the endpoint on its own. When you create a key, the dialog shows these two blocks with your values filled in.

Add the key to ~/.aws/credentials:

[coritan]
aws_access_key_id = YOUR_ACCESS_KEY_ID
aws_secret_access_key = YOUR_SECRET_KEY

Add the endpoint to ~/.aws/config:

[profile coritan]
region = fra
endpoint_url = https://s3.fra.coritan.com:7337
s3 =
  addressing_style = path

Then name the profile in each command:

aws --profile coritan s3 ls s3://u7-assets/
aws --profile coritan s3 cp ./report.pdf s3://u7-assets/reports/report.pdf
aws --profile coritan s3 sync ./site/ s3://u7-assets/site/

An older AWS CLI ignores endpoint_url in a profile and sends every request to Amazon. Upgrade the AWS CLI, or add --endpoint-url https://s3.fra.coritan.com:7337 to each command. With a custom endpoint, the AWS CLI uses path style on its own.

The Connect a client card on the Overview tab uses aws configure set, which writes the key into your default profile and replaces any key that is already there. Use the named profile above to keep your other credentials.

Create a remote called coritan:

rclone config create coritan s3 provider=Other \
  endpoint=https://s3.fra.coritan.com:7337 region=fra \
  force_path_style=true no_check_bucket=true \
  access_key_id=YOUR_ACCESS_KEY_ID secret_access_key=YOUR_SECRET_KEY

The command writes this section to your rclone configuration file, which rclone config file locates:

[coritan]
type = s3
provider = Other
endpoint = https://s3.fra.coritan.com:7337
region = fra
force_path_style = true
no_check_bucket = true
access_key_id = YOUR_ACCESS_KEY_ID
secret_access_key = YOUR_SECRET_KEY

no_check_bucket = true stops rclone from trying to create the bucket before it uploads, which an access key is not allowed to do.

rclone ls coritan:u7-assets
rclone copy ./photos coritan:u7-assets/photos
rclone sync ./site coritan:u7-assets/site

rclone sync makes the destination match the source, so it deletes objects that the source does not have.

For a single command, pass the settings as options:

s3cmd --access_key=YOUR_ACCESS_KEY_ID --secret_key=YOUR_SECRET_KEY \
  --host=s3.fra.coritan.com:7337 --host-bucket=s3.fra.coritan.com:7337 \
  --ssl ls s3://u7-assets/

To keep them, put them in ~/.s3cfg:

[default]
access_key = YOUR_ACCESS_KEY_ID
secret_key = YOUR_SECRET_KEY
host_base = s3.fra.coritan.com:7337
host_bucket = s3.fra.coritan.com:7337
use_https = True
signature_v2 = False

host_bucket holds the endpoint's host and port with no %(bucket)s in it, which makes s3cmd use path style.

s3cmd ls s3://u7-assets/
s3cmd put ./report.pdf s3://u7-assets/reports/
s3cmd sync ./site/ s3://u7-assets/site/

With boto3, pass the endpoint, a region and path style when you create the client:

import boto3
from botocore.config import Config

s3 = boto3.client(
    "s3",
    endpoint_url="https://s3.fra.coritan.com:7337",
    region_name="fra",
    aws_access_key_id="YOUR_ACCESS_KEY_ID",
    aws_secret_access_key="YOUR_SECRET_KEY",
    config=Config(signature_version="s3v4", s3={"addressing_style": "path"}),
)

s3.upload_file("report.pdf", "u7-assets", "reports/report.pdf")

listing = s3.list_objects_v2(Bucket="u7-assets", Prefix="reports/")
for item in listing.get("Contents", []):
    print(item["Key"], item["Size"])

url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": "u7-assets", "Key": "reports/report.pdf"},
    ExpiresIn=3600,
)
print(url)

upload_file sends a large file in parts on its own. Keep the secret key out of your code: leave out the two key arguments, and boto3 reads the key from the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables. boto3.Session(profile_name="coritan").client("s3", endpoint_url="https://s3.fra.coritan.com:7337") uses the AWS CLI profile instead.

Every AWS SDK takes the same settings. Only the name of the path style option differs:

• AWS SDK for JavaScript v3: forcePathStyle: true in the S3Client options.
• AWS SDK for Go v2: o.UsePathStyle = true in the options function you pass to s3.NewFromConfig.

import { S3Client } from "@aws-sdk/client-s3";

const s3 = new S3Client({
  endpoint: "https://s3.fra.coritan.com:7337",
  region: "fra",
  forcePathStyle: true,
  credentials: { accessKeyId: "YOUR_ACCESS_KEY_ID", secretAccessKey: "YOUR_SECRET_KEY" },
});

The client lists, uploads, downloads and deletes objects within the key's scope and permissions. When a request fails, the error names the reason, and Troubleshoot Object Storage explains the common ones.

The client cannot resolve, or cannot verify a certificate for, a host such as u7-assets.s3.fra.coritan.com

The client uses virtual-hosted style. Turn on path style, as each setup above does.

The client reports a certificate error for the endpoint's own host, such as s3.fra.coritan.com

The endpoint has no port, so the client connects to port 443, which serves other Coritan products. Add :7337 to the endpoint.

The connection times out

A firewall between your client and us blocks port 7337. Run curl -I https://s3.fra.coritan.com:7337. Any HTTP status in the answer, even 403, means your network reaches us, so check the endpoint in the client. No answer means something on your network blocks the port.

InvalidAccessKeyId

The key is new and has not reached storage yet, so wait a couple of minutes. Otherwise, the key was revoked, the access key ID is wrong, or the client sent the request to Amazon because it has no endpoint.

SignatureDoesNotMatch

The secret key is wrong, or only part of it was copied. Paste it again from where you saved it. If you lost it, create a new key. The same error appears when the client sends the request to an address other than the endpoint, such as an IP address, or through a proxy that changes the Host header. Use the endpoint exactly as the dashboard gives it, port included.

NoSuchBucket

The bucket is in another region, or the name lacks your prefix. Use the endpoint of the bucket's region and the full name, such as u7-assets.

AccessDenied

The key's scope or permissions do not cover the request. A read-only key cannot upload or delete, a key limited to one bucket cannot reach another, and no key can create a bucket. With rclone, set no_check_bucket = true.

• Create and revoke access keys
• Upload, download and delete objects
• Object Storage limits
• Troubleshoot Object Storage

GET /api/v1/client/object-storage/{service_id} returns the service's endpoints in endpoints, one for each region it uses (Object Storage). Each bucket in List buckets carries the endpoint of its region and its url, which is the endpoint with the bucket's name added.