Detecting the onset of covid-19 early using heartrate measurements collected from wearable devices

Through this blog post I shall be creating a solution architecture for the following Collecting wearable devices data from multiple users. For example heartrate from Fitbit and Apple HealthKit devices. Ingesting this data into an influxdb database running inside a docker container Running a pyspark job on aws EMR cluster to calculate heart rate variability,Continue reading “Detecting the onset of covid-19 early using heartrate measurements collected from wearable devices”

Python based framework for FOTA (Firmware Over The Air)

A very common requirement in IoT based systems is to remotely install updated versions of firmware on devices. Presented in this blog is a python based generic framework for performing the update. Salient features : Meager code footprint of less than 4 KB. Secure download from s3 bucket using access key and secret. Secure deviceContinue reading “Python based framework for FOTA (Firmware Over The Air)”

Creating a monitoring dashboard using Prometheus and Grafana

Prometheus is a monitoring platform that collects metrics from monitored targets by scraping metrics HTTP endpoints on these targets. Grafana is a graphical tool that connects to Prometheus and helps build a visualization dashboard. In this blog We shall spinup Prometheus as a docker container on a Ubuntu based server in Azure cloud. Install NodeContinue reading “Creating a monitoring dashboard using Prometheus and Grafana”

Load testing Python Flask based REST api server from python REST clients – AWS Autoscaling use case

Objective : Create a get, put and post api using python based flask framework Invoke the api using a python based rest client Set up an AWS load balancer in front of the server Launch a swarm of 1000+ clients to invoke the rest api from a python client using aws EC2 Auto Scaling featureContinue reading “Load testing Python Flask based REST api server from python REST clients – AWS Autoscaling use case”

Autoscaling a Kubernetes cluster using Horizontal Pod AutoScaler (HPA)

In the blog I shall be describing the step by step process to setup a Kubernetes cluster using minikube to run a node js based application in Azure cloud and autoscaling the application using Horizontal Pod AutoScaler. Spin up a Standard D4s v3 (4 vcpus, 16 GiB memory) Virtual machine in Azure based on LinuxContinue reading “Autoscaling a Kubernetes cluster using Horizontal Pod AutoScaler (HPA)”

Two ways to talk to Azure IoT Hub

This article lists down 2 ways to communicate with Azure IoT hub : An insecure method using a standalone java client A secure certificates based method using a standalone python client Most of these instructions can be found on the Azure site but is scattered all over the place. This code has been been runContinue reading “Two ways to talk to Azure IoT Hub”

Docker to Docker networking between TCP Client and Server

A TCP based client and server is a frequently needed setup in IoT based applications. Presented in this article is : A method to create a java based client and server to send and receive binary data The client and server each run inside a docker container and communicate through docker networking. Create a TCPClient.javaContinue reading “Docker to Docker networking between TCP Client and Server”

Mqtt-Based Data Transfer With Kafka Cloud

Through this article, I would like to list down the detailed steps for communicating from a device to a server in the cloud running a scalable Kafka infrastructure and back from Kafka to the device using Mosquitto broker and Kafka connect. Device to cloud connectivity using mqtt-mosquitto broker and kafka connect : Step 1: Download and install mosquito forContinue reading “Mqtt-Based Data Transfer With Kafka Cloud”

Steps to setup an Opensource Real-time IoT data pipeline in Azure cloud

Architecture diagram: Through this and a subsequent series of articles I would like to layout a detailed step-by-step process for building a realtime IoT data pipeline based primarily on Apache open source technologies on Azure cloud. I start with the detailed architecture and instructions to build production grade clusters. In the subsequent article we shallContinue reading “Steps to setup an Opensource Real-time IoT data pipeline in Azure cloud”