It's hard to say these days

A SimpleDB Critic

clock December 18, 2007 05:30 by author Tom


Hopefully this will be my last SimpleDB post for a while since I think I've spent way too much time on this(especially for a guy who should be laying off the blogging this week because he has way too much stuff to do already).  But I felt the need to quote this post from  It says...

One of the stupid parts about SimpleDB is XML. All of the query responses come back to the client as a blob of XML. Of course, as a blub coder, you don't see any of this because you are just using an API, so why should you care?

Let's have a gander at a typical query response from SimpleDB, provided in the developer documentation:

<QueryResponse xmlns=""> <QueryResult> <ItemName>eID001</ItemName> <ItemName>eID002</ItemName> <ItemName>eID003</ItemName> </QueryResult> <ResponseMetadata> <RequestId>c74ef8c8-77ff-4d5e-b60b-097c77c1c266</RequestId> <BoxUsage>0.0000219907</BoxUsage> </ResponseMetadata> </QueryResponse>


Well, that's just fantastic. A 316-byte response when all you need is 18 bytes worth of data (the 3 unique identifiers). A 1,755% transmission overhead. And don't you give me shit about needing that RequestID, that is a consequence of the protocol.

Now that is a good point.  Looking over my last few posts on this topic I think I come across as a SimpleDB advocate but that isn't necessarily the case.  All I've been trying to say in my previous posts was that the criticisms should be fair.

Anything has advantages and drawback and that is more true of SimpleDB than it is of most things.  I'm all for putting the negatives out there as long as they're valid.  The above quote is just that which is why I wanted to put it out there.

On being a long term resident of Userland

clock December 17, 2007 23:25 by author Tom

I wanted to give a quick take on a post made by Ryan Tate entitled "Babes in Userland".  The post is honestly more free-form thought than it is an article with a single point in that Mr. Tate begins by saying he thinks SimpleDB is a clunker but then goes on to praise Amazon for its web service strategy thus far (pre-SimpleDB) and finally ends up making a point unrelated to SimpleDB whatsoever. 

It is that point that I wanted to address.  To quote...

Amazon is a company that sells books and a bajillion other things over the Internet. That's what it does. Retail. It is not a software company, not even today, despite EC2 and S3, despite the fact that it has had kick-ass coders since its launched, despite what it may yet become.
Amazon is a user selling the software it made for itself.

This is important. This is new.

Microsoft and Google have always dog-fooded their own software, but that's the exception that proves the rule, which is this: Software companies make software to solve other peoples' problems.
And software companies never understand the problems they are solving as well as they understand the problem of making software.

This is flawed logic in my opinion.  It assumes that companies are monolithic, singular minded entities and that the people within those companies are all just cogs in a system that works flawlessly.  As anyone who has ever worked in a company will tell you nothing could be further from the truth. 

The truth is that companies are just tiny microcosms of the industry as a whole.  To give an example, Microsoft makes tools for developers who are their customers and they understand their customers about as well as they can through interviews and various usability tests. 

But then those developers turn around and make software for their internal employees who are essentially their customers and those developers are no more informed about their employees needs than the Microsoft person was about the developers needs.

So there really isn't that much difference between Amazon's IT people (who probably know little about actually selling books, CDs, etc...)  making software for their internal employees and Microsoft making a web service for an external customer. 

I'm going to have to ask for a little indulgence here because this is a really important point and one that is intimately related to the purpose of this blog but that is a much longer discussion and will have to be saved for another day.  For the time being just remember that any company large enough to do more than install and maintain pre-packaged software probably has a little software company tucked away inside it that works under most of the same rules and limitations as any external software company.  To think otherwise is to give Corporate America and its ability to communicate internally way too much credit. 

SimpleDB Follow Up

clock December 16, 2007 20:35 by author Tom

A little follow up on the SimpleDB discussion and the various perspectives that have come out of the web since it was introduced.

First Marcelo Calbucci makes a good point which is that SimpleDB is not in fact a database it is a directory service.  As someone who maintains an Active Directory based network I can tell you there are some significant differences in the concepts.  That said, I'll let Marcelo tell you himself...

First of, each object (this is what a "record" is called on a Directory Service) can contain different attributes and the schema can be changed on the fly (a bit more complicated than that).

    The next interesting aspect is that a single attribute (field) can have multiple values, just like the Amazon SimpleDB! This means if I define attribute "UsedBy" I can set the values to "Realtors" and "Brokers". On traditional relational databases you'd need 3 tables to do something like this.

    Finally, a Directory Service allows a hierarchy of objects, meaning instead of Tables you have nodes (which are container objects) and objects hang out of those nodes. Oh oh, SimpleDB doesn't have that, so all my theory goes down the drain.... Not really, they provide a thing called "Domain" which, if you want to (but you don't), can be used as a hierarchy.

I think this seems like a bigger distinction than it actually is in that most applications will interface with the system in the same way they would with a database.  Why I think this is such an important point though is because startups might be thinking they can sign on to SimpleDB and then just move over to a SQL Server when they are ready (I made the same point in my previous post).  That's true if you don't use some of the directory specific features but you'll need to be mindful in every aspect of your application design if you want to keep that avenue open. 

Building on SimpleDB without using the Directory specific features is going to be difficult given you also don't have things like Table Joins available to you. 

That said I think some people are making too big a deal out of this problem.  To quote Jonathan Boutelle...

Now we know. This isn't a vanilla mysql clustering service: it's something a little weirder (it's conceptually similar to a database, but lacks many of the features of a database, and works somewhat differently). As a result, you'll have to build your app from the ground up as an Amazon app: this isn't a drop-in replacement for mysql cluster.

I guess it depends on what type of application builder you are in the first place but in most cases you won't have to build things from the ground up.  This type of situation is really where the beauty of tiered design comes in.  I've not had time to study SimpleDB in detail but off the top of my head it seems like you could reproduce the concept of a database by splitting your data tier in two and having the upper level mimic database functions.  I know in .Net it would be as simple as creating a dataset and making your requisite joins in there (which isn't to say that building a solution like that would be simple just that its conceptually a simple idea to understand)

Which leads me to my last quote from Justin Rudd.  Justin says in his post that he doesn't think using SimpleDB is a good idea for startups because you are putting the responsibility for scaling completely on Amazon and if your start up gets bigger than they can support you'll end up shooting yourself in the foot.  He goes on to say...

Amazon should be part of your backup, not your front line.  Now as with all things, this is shades of gray (there is no black or white in anything in life).  If it is just you (and maybe a buddy), by all means use the Amazon Web Services to get a proof of concept up.  Get an AMAZING front end out that engages customers and keeps them coming back.  Have a couple of small web servers that you can host yourself, and let Amazon deal with all the meaty problems until you start making money.  But once you’ve proven that you’ve got a good idea, start looking at what pieces need to be brought into your own data center to guarantee a good customer experience.

Well again it comes down to the basic concept that a tiered application should have no idea where its data is coming from except to say "from the data layer".  So no complete rebuilds should be required.  You should be able to, with very little effort, transition to your own servers when it becomes cost effective to do so.  As low as Amazon's prices are they are still quite a bit higher than maintaining your own servers if you can afford to support a full time position to do that. 

That said anyone who claims you should have your own servers from the second you start making money needs to look into the time and effort it takes to maintain a database cluster (sorry Justin).  Administrating a SQL cluster is hard enough as it is and that in itself generally requires a full time person (don't expect your developers to be able to do it).  Moreover if you are a serious startup that job is going to require several full time people because you have to have someone watching it at all times.  What happens if it goes down at 1am?  The Internet is worldwide so that's the middle of the afternoon for some countries, do you really want to have 3 hour outages while your DBA wakes up, gets dressed and runs down to find out what has gone wrong?  Amazon provides a whole infrastructure for chump change that comes with built in piece of mind.  That is worth a lot to a startup even if they have started to make a little money and I don't see why they wouldn't take advantage of that in their infancy.  No one is saying a company like Facebook should be trying to run off SimpleDB.

On that note I do think companies such as Microsoft who have existing database products should work at creating a service like this that would have an upgrade path to their brand of full blown server products.  If Microsoft were to have a "SQL Server Cloud" product that allowed for easy export to a full SQL Server as a startup grows I think they might have a chance at capturing a big part of the startup market (which at this point avoids them like the plague). 

Now THAT IS Interesting...

clock December 14, 2007 18:53 by author Tom

From Joe Gregorio's site...

Amazon has opened SimpleDB for beta testers. As you would expect the system is designed to scale and as such has dropped generic relational operations such as joins and transactions. What we're looking at here is another instantiation of megadata.

And before you start beating your chest and exclaiming how "real" applications need joins and transactions please read "The End of an Architectural Era (It's Time for a Complete Rewrite)" (pdf).

His post is very short but that is where I first saw the news so I wanted to pass on the link love.  For a more descriptive post (that isn't the stock Amazon site) you can look over on O'Reilly's site

Normally I like to give the posts on this blog a little more thought than I've given this one (I just read the news a couple minutes ago) but this was so interesting to me that I had to post.  I really think Amazon is on to something here. 

This service seems to be built around (a) emulating a SQL environment and (b) providing an extremely scalable solution for start ups.  If it delivers on its performance promise this could very well convince people to stop using mySQL altogether.  If that happens it could very well rule the startup market making today a big day in the technology world.

Amazon's web service strategy has been questioned by investors since day one but just think of the money at play if they could become the de facto data provider for most of the major start ups in the world.

That kind of business would likely dwarf even their retail operations in revenue.

Addendum: Here's another interesting link.

Kindle and the Boston MP3 Party

clock November 21, 2007 14:03 by author Tom
I’ve been keeping track of the Kindle reviews (mostly through Scoble’s link blog) and for the most part my opinion of the device hasn’t changed.  The price still seems high, the wireless download (without contract or fee of any kind) still seems cool and in the end I still think it will be a failure.   That said there was something that I read in this review by Smugmug’s Don MacAskill that sparked some thought in my mind.  Here’s the quote… 
The book selection sucks. There are big gaps, even for well-known bestselling books. Having worked in the book industry before, I put most of this blame in publisher’s laps. They’re just a nightmare to deal with, and paranoid about their content. Apparently they don’t want my money or yours, and even Amazon doesn’t have the weight to make them see reason. Shades of the music, TV, and movie industries, anyone? This must be incredibly frustrating to Jeff and everyone else at Amazon.

 Now selection is of the utmost importance to the Kindle’s success and I think everyone realizes that so I won’t spend any time on it.  What interests me is the idea that the publishers are holding up the process.   

There has long been a debate as to how much influence illegal file sharing had on the music industry and specifically on the industry’s caving in to services like iTunes.  This reaction from the publishing industry seems to point to the idea that file sharing was instrumental in turning the tide towards digital media.  Look at the facts, here we have an industry that basically works under the same principles (e.g. selling content on physical media) but which sells a product that is hard to reproduce in digital form (scanning a book is much harder than ripping a CD) and that industry is fighting tooth and nail to not go digital (hence the lousy selection).    

So the question I now have is “does this justify illegal trading?”  I’m no radical but any historian will tell you that breaking the law was sometimes necessary to bring about change (see the Boston Tea Party allusion above).  Maybe it’s just my old guilt over file trading as a kid but as someone who has bought his music legally since that has been an option I feel a little validated by this. 

Amazon's Kindle

clock November 19, 2007 13:32 by author Tom

Leave it to the blogosphere to bash a device before its even been announced but that is what's happening to the poor Amazon Kindle e-book reader. 

Jeremy Toeman asks "Is the market today "in crisis" when it comes to books?" and then supplies the answer "No".  I would disagree in that I personally would do anything to have my library of computer books with me both at work and at home but carrying three bookshelves isn't plausible.  I don't think I'm a niche in this either.  To me Jeremy saying there is no crisis here sounds a lot like the people who asked "why would I want to carry all my music around with me?" when the iPod first arrived.

Jeremy does go on to make some decent points about e-book readers but since his post was contingent on there being no great advantage to the e-book reader the remaining points fall pretty flat for me. 

I found myself agreeing with Rex Hammock who says the iPod Touch is a  much better model for an e-book reader but I'm not sure that automatically makes the Kindle a bad product.  The iPhone/iPod Touch model is revolutionary and its going to put almost any other handheld device to shame as far as things like navigation are concerned.  But that's no reason to necessarily dismiss a product that has other advantages (Such as allowing you to wirelessly purchase e-books from anywhere and have access to them that instant). 

Plus, my understanding is that the Kindle does a good job of replicating paper with its screen and that shouldn't be overlooked.  Whether you realize it or not looking at a computer screen is hard on the eyes and a decent e-paper solution is worth a lot in the battle to replace the printed book. 

All that said, I haven't quite formed an opinion on the kindle yet.  There are certainly points that I like (mostly the wireless download) and things I don't (high price, appearance) but it really is too early to tell.  If I had to guess right now I'd bet against it but I also would have bet against the original Palm Pilot and look how that turned out. 

One last note, the thing looks really ugly which doesn't help.  I can't help but think Amazon would have been better off had they made something that looked a little more like this. 

About Me

Not really relevant right now. This blog is on hiatus. I really haven't decided if it is an indefinite hiatus yet

For the record if you've tried to e-mail me over the last 4 to 6 months I didn't mean to ignore you. The e-mail forwarding isn't working and I didn't realize that until months worth of e-mails had been deleted on forward. The address still won't forward to the postmaster account and I don't know why because it's provided by the webhost. But if you're one of my old blog pen pals I would always welcome an e-mail from you at the address


- E-Mail Tom



- Subscribe to this Blog


<<  May 2016  >>






    The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

    © Copyright 2016

    Sign in