Talking to RADOS Gateway¶
Using Swift¶
Install a swift command line client, such as python-swiftclient:
$ sudo yum install python-setuptools
$ sudo yum easy_install pip
$ sudo pip install --upgrade setuptools
$ sudo pip install --upgrade python-swiftclient
This allows you to use the command line syntax:
$ swift -A http://[ENDPOINT_IP]/auth/1.0 -U [USER:SUBUSER] -K [KEY] [COMMAND]
There are slight differences to the syntax depending on whether you are using the radosgw’s built-in authentication, or Keystone authentication.
Swift with Keystone authentication¶
Assuming $KEYSTONE_ENDPOINT, $OS_PROJECT, $OS_USERNAME, and $OS_PASSWORD are all set with your openstack credentials:
$ swift -V 2.0 -A $KEYSTONE_ENDPOINT -U $OS_PROJECT:$OS_USER -K $OS_PASSWORD list
The output should show a list of containers in your account.
This longer syntax also works:
$ swift -V 2.0 --os-auth-url $KEYSTONE_ENDPOINT --os-username $OS_USER --os-password $OS_PASSWORD --os-tenant-name $OS_PROJECT list
Swift without Keystone Authentication¶
If you are using accounts created using the radosgw-admin CLI interface, you need to use the radosgw address and your user:subuser and key credentials
$ swift -A http://rdgw.kaizen.massopencloud.org/auth/1.0 -U kamfonik:swift -K "ABCDEFG1234567890" list
will output:
testbucket
testbucket2
(assuming the subuser kamfonik:swift has these two top-level buckets)
To see what’s in a bucket, add the bucket name to the end of the command:
$ swift -A http://129.10.3.96/auth/1.0 -U kamfonik:swift -K "ABCDEFG1234567890" list testbucket
which outputs
sub_testbucket
Run man swift to see what other commands are used for creating, deleting, and manipulating objects.
Using Curl¶
You can also talk to the radosgw without a swift client, using curl.
Keystone Authentication¶
First, get a token from keystone:
$ AUTH_DATA='{"auth": {"tenantName": "'"$OS_PROJECT_NAME"'", "passwordCredentials": {"username": "'"$OS_USERNAME"'", "password": "'"$OS_PASSWORD"'"}}}'
$ curl -i -H "Content-Type: application/json" -d "$AUTH_DATA" $OS_AUTH_URL/tokens
(The quotes around $AUTH_DATA are important, otherwise you will get an error that your JSON is not valid.)
Get the token ID from the response JSON. Use the token ID to authenticate to the radosgw directly:
$ MYTOKEN=<put the token ID string here>
$ curl -X GET -H "x-auth-token: $MYTOKEN" http://rdgw.kaizen.massopencloud.org/swift/v1
The above GET command outputs a list of top-level containers. More info about the different API calls and how to construct them can be found in the official Ceph Documentation.
You could do something similar with Python’s requests library or similar as needed.
Authentication without Keystone¶
Before running any commands, you must authenticate with the gateway:
$ curl -i -H "X-Auth-User: [USER:SUBUSER]" -H "X-Auth-Key: [KEY]" [URL]
Example:
$ curl -i -H "X-Auth-User: kamfonik:swift" -H "X-Auth-Key: LNL+8e8K8H8hCxajNovT0FWmFwZxnENo3nMFPHcX" http://129.10.3.96/auth/v1.0
The reply will be a storage URL (X-Storage-URL) and an authorization token (X-Auth-Token):
HTTP/1.1 204
Date: Thu, 05 Nov 2015 22:45:35 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux)
X-Storage-Url: http://129.10.3.96/swift/v1
X-Storage-Token: AUTH_rgwtk0e0000006b616d666f6e696b3a7377696674e5c40bcea38082df8f2d3d56c07ee422bd169b60c15b868e65d569c9371569331e2620b0
X-Auth-Token: AUTH_rgwtk0e0000006b616d666f6e696b3a7377696674e5c40bcea38082df8f2d3d56c07ee422bd169b60c15b868e65d569c9371569331e2620b0
Content-Type: application/json
For the rest of the steps, you will direct your commands to this URL, and use this token to authenticate.
For convenience, you can put the url and token in variables:
MYURL="http://129.10.3.96/swift/v1"
MYTOKEN="AUTH_rgwtk0e0000006b616d666f6e696b3a7377696674e5c40bcea38082df8f2d3d56c07ee422bd169b60c15b868e65d569c9371569331e2620b0"
Note: the token may expire after an interval and you will need to authenticate again to get a new one.
Here are a few examples of commands and their output:
Create Bucket with Curl¶
$ curl -i -X PUT -H "X-Auth-Token: $MYTOKEN" "$MYURL/testbucket2"
Output:
HTTP/1.1 201
Date: Thu, 05 Nov 2015 22:46:49 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux)
Accept-Ranges: bytes
Content-Length: 0
Content-Type: text/plain; charset=utf-8
List Buckets with Curl¶
$ curl -i -X GET -H "X-Auth-Token: $MYTOKEN" "$MYURL"
Output:
HTTP/1.1 200
Date: Thu, 05 Nov 2015 23:08:08 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux)
Transfer-Encoding: chunked
Content-Type: text/plain; charset=utf-8
testbucket
testbucket2
Delete Buckets with Curl¶
$ curl -i -X DELETE -H "X-Auth-Token: $MYTOKEN" "$MYURL/testbucket2"
HTTP/1.1 201
Date: Thu, 05 Nov 2015 Nov 2015 23:16:49 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux)
Accept-Ranges: bytes
Content-Length: 15
Content-Type: text/plain; charset=utf-8
Sub-Buckets with Curl¶
To create, list, or delete lower-level buckets, append name(s) of the outer bucket(s) to the storage URL:
$ curl -i -X GET -H "Content-Length: 0" -H "X-Auth-Token: $MYTOKEN" "$MYURL/testbucket"
HTTP/1.1 200
Date: Thu, 05 Nov 2015 Nov 2015 23:22:42 GMT
Server: Apache/2.4.6 (Red Hat Enterprise Linux)
Accept-Ranges: bytes
Content-Length: 15
Content-Type: text/plain; charset=utf-8
sub_bucket_test
Using Amazon S3 interface with Python-Boto¶
Amazon S3 With Keystone Authentication¶
Coming Soon!
Amazon S3 Without Keystone Authentication¶
First, install python-boto:
$ sudo yum install python-boto
Then, create a test script called s3test.py with these contents:
import boto
import boto.s3.connection
access_key = [S3 ACCESS KEY]
secret_key = [S3 SECRET KEY]
conn = boto.connect_s3(
aws_access_key_id = access_key,
aws_secret_access_key = secret_key,
host = '[RADOS GATEWAY FQDN]',
is_secure=False,
calling_format = boto.s3.connection.OrdinaryCallingFormat(),
)
bucket = conn.create_bucket('testbucket')
for bucket in conn.get_all_buckets():
print "{name}\t{created}".format(
name = bucket.name,
created = bucket.creation_date,
)
Run this script to create a bucket called testbucket, and list all buckets.
Script code for deleting testbucket would be:
conn.delete_bucket('testbucket')
