This is the second entry in a series (prior entry here) explaining how you can use Cloud Sidecar to take your software and run it multiple clouds with very little code change. What is Cloud Sidecar? It is an application that runs next to your software as a sidecar, intercepts cloud API requests, and converts the API calls to a different cloud provider without your software noticing. Since it intercepts API calls, it works for virtually any software or language. In this entry I will show you how to convert your software from using Amazon SQS to Google Pub/Sub
How To Do It
Make sure you have Golang installed, preferably 1.15.x or higher
Clone Cloud Sidecar from github
./build.sh to build the binary for your target architecture
Create a config file named config.yaml. It should look something like
aws_configs: sqs: service_type: "sqs" port: 3460 aws_destination_config: name: "example" access_key_id: "bleh" secret_access_key: "bleh" gcp_destination_config: name: "example" key_file_location: "/location/to/gcp/creds.json" project: "project name here" pub_sub_config: read_timeout: "10s"
Alter your application code to connect to connect to localhost on port 3460. Here are examples on how to do it in Python and Java
import boto3 import os client = boto3.resource( "sqs", region_name="us-east", endpoint_url='http://localhost:3460', ) queue_url = "https://sqs.us-east-1.amazonaws.com/someid/queue-url" response = sqs.receive_message( QueueUrl=queue_url, AttributeNames=[ 'SentTimestamp' ], MaxNumberOfMessages=1, MessageAttributeNames=[ 'All' ], VisibilityTimeout=30, WaitTimeSeconds=5 ) message = response['Messages']
import com.amazonaws.services.sqs.AmazonSQSClientBuilder; import com.amazonaws.services.sqs.model.AmazonSQSException; import com.amazonaws.services.sqs.model.SendMessageBatchRequest; final AmazonSQS sqs = AmazonSQSClientBuilder .standard() .withEndpointConfiguration(new EndpointConfiguration("http://localhost:3460", Regions.US_EAST_1.getName)) .build() List messages = sqs.receiveMessage(queueUrl).getMessages();
Start up Cloud Sidecar by running
AND NOW YOU ARE TALKING TO PUB/SUB!!!
This was a very simple example, but can easily be extended to more complex applications, use Kubernetes, etc… If you have any questions feel free to reach out to me at email@example.com or drop a comment in this blog entry.