Cloud Security Architecture Matters

Not all cloud services are created equal. Clouds are architected by flesh and blood men and women, and because the are no cloud standards yet, the architectural choices are invariably different.
A couple of months ago I was looking for differences between various cloud vendors' API implementations, to show at a local OWASP meeting.

And differences I found. The most important difference is in the way users present their credentials to the service, and how the service ensures that it receives a valid request.

To make things comparable, let's look at the way 3 leading cloud vendors (Amazon Web Services, GoGrid and RackSpace) authenticate and authorize usage of their cloud resources.

We care about the way the service ensures that it receives a valid request, because we want to minimize the risk of account hijacking, and to minimize the risk of action replay.

A hijacked account means that a third party does unintended actions using resources that are tagged as belonging to the account owner. The third party can use the account for unwanted activity, such as distribution of illegal materials or promotion of spam, and the bill is handed out to the account owner.

Action replay means that someone may repeatedly send identical commands to the cloud infrastructure on behalf of the account owner. At best it is just a nuisance that may cost a little.
At worst, it may enable a third party to replace valid content with invalid, or even malicious content.

I am going to compare the basic action of listing the contents of the cloud objects container.
To make long things short, here is a brief description of the results.

• All APIs make use of the HTTP protocol, with some custom extended headers.
• All APIs require a public access identifier and a secret key shared between the account owner and the vendor.
• AWS does not need the secret key in plaintext. It requires a unique signature for each request, making it extremely difficult to do any harm to the account, even if the request is somehow intercepted on the way.
• GoGrid does not need the secret key in plaintext. It requires a signature as part of an authentication request, and returns a security token that is valid for a period of 10 minutes, and for all actions within 10 minutes. 
• RackSpace requires the secret key in plaintext as part of an authentication request, and returns a security token that is valid for a period of 24 hours, and for all actions within 24 hours.  

Backing up to the cloud

My previous post was about using the Jets3t synchronize tool on an iSeries server.
While synchronize works quite well, it is not flexible enough for what I had in mind. To be able to integrate Amazon S3 with a real life backup and DRP policy, I want a set of atomic functions that enable me to simply transfer particular data objects between S3 and the server.
Using the Jets3t rich set of S3 objects and the samples provided by the author, I found it quite easy to write additional proof of concept tools.

I now have Create Bucket, Upload Single file, and DownLoad Single File programs. These are all barebones tools, use them at your own risk..

All tools use the iUtils helper tool that provides for central management of parameters and credentials. In fact, iUtils uses exactly the same configuration file Synchronize does.

I timed the Upload and Download tools with a 500MB savefile, and on my network it took less than 30 minutes to go either way.

The usage is quite simple:

• CreateBucket requires a single parameter, the new bucket name.
• uploadSingleFile requires two parameters: the target bucket name, and the file name to upload
• DownLoadSingleFile requires two parameters: the source bucket name, and the file name to download.

Get the extended toolset files here. As I said before, these are barebones tools and there is a lot to improve. There is practically no graceful error handling, no compression and no encryption.

Using Amazon S3 in an iSeries backup/recovery scenario
S3 can be used as an offsite backup for some of your data, maybe even for all of your data. A library can easily be saved to a save file, and the save file uploaded to S3 storage until needed, at which time it can be downloaded and restored to your iSeries.

For example, look at the following set of commands that saves changed objects to a save file, zips it and sends it to S3. The extra zipping is required because the data compression built into the iSeries save commands is not very efficient, and because I have not implemented it yet as an integral part of the upload tool (although the functionality exists in the jets3t package).

/* create save file */
crtsavf s3wk200901

/* backup objects changed since December 31 to save file */
savchgobj obj(*all) lib(mylib) dev(*savf) savf(s3wk200901)
 refdate('31/12/2008') clear(*all) dtacpr(*high)

/* copy save file to IFS */
cpytostmf frommbr('/qsys.lib/mylib.lib/s3wk200901.file')
 tostmf('/backups/s3wk200901.savf') stmfopt(*replace)

/* further compress save file */
qsh cmd('jar  cMf  /backups/s3wk200901.zip /backups/s3wk200901.savf')

/* upload compressed save file to S3 */
qsh cmd('cpytos3.sh mybackup /backups/s3wk200901.zip')

Synchronize iSeries files with Amazon S3 virtual storage

Amazon Web Services, or AWS, is one of the new and hot promises to revolutionize IT.
I subscribed to AWS almost two years ago, but started to really use it only on last October.

AWS provides huge benefits to IT, developers and web site owners.
For IT, other than the ability to dynamically allocate servers and pay for the actual server utilization, AWS provides the ability to have a reliable, secure remote storage.

I went out to see whether Amazon S3 lives up to my expectations and can play a significant part in a disaster recovery plan. Some smart folks already did some math a while ago, and the economic drivers for using S3 are now higher than ever.

It turns out that there are quite a few S3 tools. For S3 management I ended up using the S3 organizer, a free FireFox extension, and the S3 Cockpit tool, which is actually part of a larger package, Jets3t.

There is a major difference between personal tools and tools appropriate to use in a datacenter. The biggest challenge for me was to find those tools that support multiple platforms, and that once configured, work in unattended mode with no hassle.

I settled for Jets3t - a set of libraries and tools written in Java.
On a windows server, installation is straightforward: just unzip the distribution archive, set up some environment variables and you are ready to start. Of course, Java is a prerequisite.

Jets3t consists of a set of libraries that wrap the S3 API with Java objects, and 4 working tools:

• Cockpit, a GUI application/applet for viewing and managing the contents of a S3 account.

• Synchronize, a console application for synchronizing directories on a computer with an Amazon S3 account.

• CockpitLite, a GUI application/applet for viewing and managing the contents of an S3 account, where the S3 account is not owned by the application's user directly but is made available via the Gatekeeper servlet.

• Uploader, a wizard-based GUI application/applet that S3 account holders (Service Providers) may provide to clients to allow them to upload files to S3 without requiring access to the Service Provider's S3 credentials.

The most promising tool, as far as I am concerned, is Synchronize.
Working in a manner similar to rsynch, this tool can keep an S3 bucket and a local folder in synch with each other, while providing both compression and encryption in the cloud.

After making it work on my PC, I set out to make it work on an iSeries server.

Setting up on iSeries
The first thing to do is to install Jets3t on your server, in any IFS directory you like.
I placed it in the /QIBM/UserData/aws/jets3t folder.

Jets3t relies on some environment variables to be defined, specifically JETS3T_HOME and JAVA_HOME.
Make sure both are defined before executing the synchronize.sh script.
I set up both as global variables by running these iSeries commands:

addenvvar JETS3T_HOME   '/QIBM/UserData/aws/jets3t'  LEVEL(*JOB) REPLACE(*YES)
addenvvar JAVA_HOME   '/QIBM/ProdData/Java400/jdk15'  LEVEL(*JOB) REPLACE(*YES)

Additionally, the unix shell scripts that came with the package assume that the shell is in /bin/sh
This probably is not true on your iSeries server, so either modify the shebang lines or run this command to solve the problem

ADDLNK OBJ('/usr/bin') NEWLNK('/bin') LNKTYPE(*SYMBOLIC)

Synchronize supports file compression and encryption, as well as various modes of operation that make synching a breeze. For example, to synch the /home/shalom/personal folder with a S3 bucket called shalom-personal, you will execute

synchronize.sh UP shalom-personal /home/shalom/personal/

You do not have to create the bucket prior to using synchronize, it will create the bucket if it does not exist. ( tip: use the --noaction switch to verify your actions before commiting to any changes. )

To learn how to actually use synchronize, read the manual.

To make it easier on the iSeries folks, here is a CL program and CMD that wrap the synchronize tool in a familiar interface.

In the next post I will explain how to extend Jets3t to a real on-demand storage device with some Java programming.