(2021-10-28) Bray URI-based Integration

Tim Bray: URI-based Integration. If you need to integrate an arbitrarily large and and diverse set of software capabilities, URIs are the best integration glue.

Background · AWS Step Functions launched in December 2016, and I did a whole lot of work on it. In particular, my fingerprints are all over the Amazon States Language, a JSON DSL that describes all the workflow stuff

*The argument · When we first cooked up the product, the only real target we had was Lambda functions, and so the suggestion was that we have a "Type": "Lambda" state, with another field that would give the name of the Lambda function.

But I said “Long-term, we want to be able to orchestrate lots of other things, not just Lambdas, right?” Everyone agreed. So I said “OK then, let’s just have a Task state which identifies the worker with a URI. That way, everything we orchestrate has the same contract, you send it some JSON and you get some JSON back.”*

People looked a bit puzzled and said “But Lambdas don’t have URIs.” I said “Sure they do, they have ARNs and ARNs are URIs.” (Well, they would be if Amazon registered the “arn:” URI scheme

I was remembering the dawn of the Web, quoting from someone (I think TimBL?) who said “On the Web, a resource is a unit of information or service.” Which I thought was a good fit here

As I write this, there are 17 “Optimized” integrations, and then 200+ SDK-based integrations. And they all use the same Task state and address the target worker by URI (which at the moment is always an ARN).

The recent announcement that kicked this discussion off made it possible to call more or less any API in the AWS SDK, addressing it with an ARN.

I’m feeling just the tiniest bit smug that they were able to add all these integrations, and in particular this latest huge one, without needing to make any major changes to the States Language.

I’ve always thought that once you agree to address things by URI, well that includes HTTP URLs, so why shouldn’t a Step Functions Task state be able to include an arbitrary external Web endpoint? SNS can already do this.