Hello World with a basic Kafka Producer and Consumer. It is assumed that you know Kafka terminology. There is a lot to learn about Kafka, but this starter is as simple as it can get with Zookeeper, Kafka and Java based producer/consumer.
Most serious applications (and distributed microservices style architectures) will require to provide a log aggregation & analysis feature to its dev & operations teams. Reviewing log entires from 10s or 100s of server instances is not something to take lightly. Whether you choose to use a commercial product or an open source offering – that does not matter; just make sure you have one available.
Recently I have been deploying applications using AWS Beanstalk. You can definitely configure CloudWatch Logs to send log streams over to AWS ElasticSearch service. Log messages can be routed to a Lambda function which would break the log messages into individual attributes suitable for indexing. I wanted to try a slightly different route where I depend less on CloudWatch Logs and more on open source tools. Enter filebeat on Beanstalk.
Just came across a nice dev tool to build and deploy Docker’ized applications. A developer may want to independently build/deploy to a local docker container or another Kubernetes cluster vs. waiting for a remote CI/CD process to deploy to a dev environment. This involves some repetitive tasks such as building the binary, packaging into a Docker container, pushing to a Docker registry and all the way to deploying to a running container. If you run this often it can be a time suck for developers. forge.sh takes away those repetitive steps and lets you build/deploy locally or to another cluster.
Have you worked in a high performance engineering (HiPeE) team, how do you know you are in one and why is it so hard to replicate that? Are there factors that make it possible for HiPeE teams to grow and sustain. This was a topic of discussion at a recent chat with a few folks. I speak from my own experiences having seen HiPeE teams and was lucky enough to be part of it couple of times. It came down to a few dimensions for me.
Attended my first AWS re:invent conference. It was definitely a great experience. While the breakout sessions where okay, what I most enjoyed is the energy with 30k AWS enthusiasts in the building. And the re:play party was awesome – especially the performance by Martin Garrix.
Notes on getting Docker Swarm cluster running on two Raspberry Pi 3 devices. We will then deploy a simple nodejs app container on the new cluster.
Updated one of my previous Spring Boot sample service to run within a Docker container – https://github.com/thomasma/quote-service-docker. You can run it locally w/o Docker as a regular Spring Boot app and next run it inside a Docker container. Make sure that you have Docker setup correctly and tested prior to running this app.
It is hard not to be affected by the constant chatter on Microservices Architecture and Container technology. Both are leading the discussions nowadays and they combine to provide new ways to Architect distributed systems and provide agility in delivering business value. While they do bring in big benefits when implemented successfully, the path to success for most enterprises (other than startups/product/tech firms) is going to be difficult and having a level of measured caution would be good.
Random thoughts on mobile from a presentation slide I created a while back and some updates thoughts based on new information from talking to another expert.
To discuss Serverless Architecture we need to understand how we got here. From using physical machines we moved to virtual machines (somewhere in between a few brave folks also used linux/solaris containers). The current trend is container technologies such as Docker or CoreOS RKT which allow even more efficient use of resources. Regardless of which you use, we are often required to plan our application infrastructure needs upfront and permanently keep the “servers” running.