Apache superset provids an official docker image available on Dockerhub https://hub.docker.com/r/apache/superset that can be used to run superset but this image lacs few functionality which are critical to use it in production environment.

Missing components from official Apache Superset docker image:

• User Login
• Database Drivers
• Certificates and other dependencies

User Login

Superset image form dockerhub does not provide any easily accessible way to configure Admin user for the app. It relies on superset fab create-admin command to create admin user. We often do not have acess to perform these tasks manually from a docker container (specially when running on a managed cloud service like AWS ECS).

Database Drivers

Apache Superset supports a wide range of databases for visualization. Official dockerhub image keeps it light by not addd all the required drivers and letting you add them later. You can find driver for your need in the list https://superset.apache.org/docs/databases/installing-database-drivers/ here.

Certificates and other dependencies

Superset supports wide range of databases and services to be integated with and lot of these services support using custom SSL Certificates to be used for the communication. Apache Superset does not comes with intutive user interface to add these certificates and other dependencies. It has added few features in recent version to add ROOT CA from UI still sometimes it’s easier to have these dependencies baked in before you run it.

Additional Configuration Support

Superset uses python file named superset_config.py to customize a lot of superset functionality. You can set many variables which are used in Superset for enabling/disabling features. Below are few examples of that are

Enabling Jinja template

FEATURE_FLAGS = {
    "ENABLE_TEMPLATE_PROCESSING": True,
}

Enabling Proxy Fix for SSL support

ENABLE_PROXY_FIX = True

Creating Custom Docker Image

Simplest solution to these problems is creating your own custom docker image. You can do it form Apache Superset source https://github.com/apache/superset or by extending official image. You can add more functionalites by adding custom config files and unix scripts that can we run an entrypoint (start of the docker container).

superset-custom.sh

#!/bin/bash

# create Admin user, you can read these values from env or anywhere else possible.
echo "Creating admin user ${ADMIN_USERNAME} email ${ADMIN_EMAIL}"
superset fab create-admin --username "$ADMIN_USERNAME" --firstname Superset --lastname Admin --email "$ADMIN_EMAIL" --password "$ADMIN_PASSWORD"

# Upgrading Supersetset metastore.
echo "Upgrading DB"
superset db upgrade

echo "Setup roles"
superset superset init  #setup roles and permissions

echo "Starting server"
/bin/sh -c /usr/bin/run-server.sh

Dockerfile

# Define base image
FROM apache/superset:1.5.1

# Switching to root to install the required packages
USER root

# Intall required dependencies.
# Example: installing the MySQL driver to connect to the metadata database
# if you prefer Postgres, you may want to use `psycopg2-binary` instead
RUN pip install psycopg2

# Define additional environment variables to be used 
ENV ADMIN_EMAIL admin@superset.com

# Add custom superset_config.py file and shell files
COPY superset_config.py /app/
ENV SUPERSET_CONFIG_PATH /app/superset_config.py #define variable that is used by Superset to look for config python file.

COPY superset-init.sh /app/superset-custom.sh
RUN chmod +x /app/superset-custom.sh

# Switching back to using the `superset` user
USER superset
ENTRYPOINT [ "/app/superset-custom.sh" ]

You can add a lot more to your docker image and customize it for you purpose. I hope it will help you starting with your Apache Superset Journey

Context

Team built an amazing service to provide ASR and TTS for indic languages. GPUs were considered to have better performance. Kubernetes chosen to keep things simple and scalable at various levels. Everything was smooth until we got to know that we have to share GPUs and kubernetes does not have native support for GPUs. K8s official documentation says

Kubernetes includes experimental support for managing AMD and NVIDIA GPUs (graphical processing units) across several nodes.

Our services were using libraries like pytorch which were using NVIDIA’s CUDA for GPU computations.

Nvidia CUDA (Compute Unified Device Architecture)

Provides abstraction and high level language support on a variety of Nvidia GPUs.

• Native CUDA programming can be done in C or C++
• Use a wrapper library like python, Java

More on it CUDA documentation.

Why Share GPUs

• GPUs are expensive.
• Better utilization of GPU cycles and memory
  • One model instance may not consume it all.
  • Availability of best fit GPUs for your workload.

We always want to get most value out of any resource that we use. Our benchmarking showed the scope to do more on one single GPU.

*Peeks in the graphs represent load. We did these runs to benchmark our ASR model service. Here, one ASR model was running on an Nvidia T4 GPU which has 16 GB memory. With our increase in load utilization increases but saturates at a level and did not increase with increase in load (load means increase in number of concurrent request).

These load runs made it clear that deploying one instance of the service (one model running in a service) per GPU wasn’t the best use of it. We can load more models on to one GPU and get better results. We also have to mindful of that fact that current generations of GPUs are not designed for parallel workloads. So benchmarking plays important role in understanding the impact.

GPU with Kubernetes

Kubernetes has no internal support for GPU resource scheduling and isolation. Instead, it is implemented by third-party through the device plugin and scheduler extension.

As per k8s documentation (https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus)

Kubernetes includes experimental support for managing AMD and NVIDIA GPUs across several nodes. Containers (and Pods) do not share GPUs. There’s no overcommitting of GPUs. Each container can request one or more GPUs. It is not possible to request a fraction of a GPU.

What you can do

resources:
        limits:
          nvidia.com/gpu: 1 # requesting 1 GPU, Your pod will see one GPU only.

This will only take care of scheduling your workload to an available GPU. It will be able to restrict utilization, memory etc. which you would usually find with other resources like memory, CPU. This feature will restrict the GPU visibility to your pod, it will only see defined number of GPUs in your pod. No other workload will be allowed to schedule on that instance with those GPUs assigned.

You cannot do this

resources:
        limits:
          nvidia.com/gpu: 0.5 # cannot request fractions.

Trick

You can set nvidia.com/gpu value to 0 and still workload will be able to see all the GPUs available on the instance. It will also not block the GPU on kubernetes to more workloads can be scheduled on that node.

resources:
        limits:
          nvidia.com/gpu: 0 # This will work fine and will not block your GPU for other workloads.

Ways to achieve GPU sharing in kubernetes

There are multiple ways to share a GPU instance with multiple processes like we do with CPU.

• Hardware level partitioning like MIG
• Virtual GPUs
• Custom implementations

Hardware level partitioning

This was our first level of option. NVIDIA provides an option of creating MULTI-INSTANCE GPU at hardware level. One single physical GPU can be mapped to machine as upto seven GPUs. This feature is limited to specific NVIDIA GPUs and need a cluster/machine level changes to support.

Virtual GPUs

This is similar to what we do we CPU but not as cool. AWS has created device plugin to support creation of VGPUs (Virtual GPU) https://github.com/awslabs/aws-virtual-gpu-device-plugin.

• We can allocate 10 virtual GPUs for each physical GPU.
• We can run more than one pod on a physical GPU.
• We can exploit multi-process service for better performance.

If you have large GPUs, you can do MIG and on top of that use device plugin.

This will require you to replace your exiting GPU driver with AWS virtual GPU driver.

Limitations: Plugin does provide any constraints over resource consumption. Process has to put limits itself.

Even in EKS this does not come out of the box as of April-2022. You have to install it on your cluster.

Custom implementations

MIG and Virtual GPU both these options require you to provision your hardware or cluster with custom drivers. In case, you do not have access to such level only choice that remain with us is GPU Sharing without Kubernetes. This can be done in multiple ways.

• Writing custom code
• Using CUDA_VISIBLE_DEVICES

Writing custom code

In case you have multiple GPUs attached to one single machine. CUDA will by default load data to 0th device. You can change that by creating a device object and passing this device object at all the places. Another challenge is deciding which GPU to use.

Challenges

• By default, CUDA uses 0th device.
• CUDA does not provide support to get memory usage of a GPU in any programmatic way.

Sample nvidia-smi output

Solution

• Change your code to load models and data on specific device.
• Parse output of nvidia-smi to see memory consumption of a GPU.

Tip for Python Users GPUtil is a package to select GPU based on your criteria.

GPUtil.getAvailable(order='memory', limit=1, maxLoad=0.8, maxMemory=0.75,excludeID=excluded_gpus)

Reference CUDA python library

Using CUDA_VISIBLE_DEVICES

CUDA provides a range of environment variable which can be used to control the visibility of GPU devices at process level. CUDA_VISIBLE_DEVICES is one such variable. When you are starting a process you can set an environment variable CUDA_VISIBLE_DEVICES with one or list of GPU ids and that process will only see those GPUs as available devices. As this is done at process leve, k8s does not know about this hence you can run multiple workload on same GPU device with your judgement of memory and utilization.

Expose 2nd GPU

CUDA_VISIBLE_DEVICES: "1"

Expose 2nd and 3rd GPU

CUDA_VISIBLE_DEVICES: "1,2"

Summary

GPU sharing is still an experimental feature in kubernetes. There are efforts going on to make it available. Till then, we have to live with workarounds like these to make it work. It requires a bit of planning upfront to know which workload will go where and what GPU instance will be used by them.

References:

CUDA Documentation

AWS Virtual GPU Plugin

NVIDIA Multi-instance GPU

Scenario

We created a GRPC service and deployed it on k8s using a ClusterIP service which was exposed using an Nginx server. We did some load tests and figured out its vertical scale limits. so we decided to scale the number of pods on our service to achieve better results. As the k8s documentation mentions

By default, kube-proxy in userspace mode chooses a backend via a round-robin algorithm

With this assumption we expected our horizontally scaled service to evenly distribute load across pods. When we did this we were surprised to see that most of the load was still going to 1 pod. We increased the number of pods but the pattern continues and the load is not evenly distributed.

Problem

After spending hours on the internet and playing with various configurations, we found that GRPC needs HTTP2 and ClusterIP was using HTTP2 which was doing multiplexing. Because of this multiplexing, it was reusing existing connections and sending new requests to the same k8s Pod rather than creating a new connection to the second pod. This phenomenon continues with more pods and a few pods that get most of the load.

Un-even Load Distribution

Solution

As we figured out the problem is with the load balancing system and a solution was achieved in two steps:

• Create a headless k8s service.
• Using a custom/external load balancer.

Let’s look at both of these options in detail

Creating headless k8s service

apiVersion: v1
kind: Service
metadata:
  name: headless-svc
spec:
  clusterIP: None # <= Don't forget!!
  selector:
    app: web
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

A Kubernetes headless service is a ClusterIP service that doesn’t create a cluster IP for your service and exposes the IPs of all the pods that are created. With this approach, you can get the IPs of all the pods that are available for your App.

Using custom/external load balancer

Once your headless service is available you can use any load balancer that has the capability to discover IPs from the ClusterIP service and do the load balancing. For us, envoy being part of our system came to our rescue. We were using envoy to support REST clients on our GRPC service.

Example Envoy Config

clusters:
  - name: headless-svc
    type: STRICT_DNS
    lb_policy: ROUND_ROBIN
    connect_timeout: 30s
    dns_lookup_family: V4_ONLY
    load_assignment:
      cluster_name: headless-svc
      endpoints:
        - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: headless-svc
                    port_value: 80

Summary

Kubernetes provides a great abstraction to our deployments, and sometimes it can make debugging load balancing and other internals of services very complex. HTTP2 brings some performance optimisation which can cause behaviours that are not good for some loads. Having an external load balancer that brings more control and clarity can be a boon. Envoy is one such proxy that can do a lot of powerful stuff with very little resources.

2020-04-21 21:19:16.829  INFO 12332 ---
[nio-8080-exec-5] Spring Security Debugger : \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* Request received for GET '/images/favicon.ico': org.apache.catalina.connector.RequestFacade@20617566 servletPath:/images/favicon.ico pathInfo:null headers: host: localhost:8080 connection: keep-alive pragma: no-cache cache-control: no-cache user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36 sec-fetch-dest: image accept: image/webp,image/apng,image/\*,\*/\*;q=0.8 sec-fetch-site: same-origin sec-fetch-mode: no-cors referer: http://localhost:8080/ accept-encoding: gzip, deflate, br accept-language: en-US,en;q=0.9,hi;q=0.8 cookie: ext\_name=jaehkpjddfdgiiefcnhahapilbejohhj; JSESSIONID=B31522BDCCAA126406166C25DBE9DE96; sub\_pop=1587484276805 Security filter chain: [] empty (bypassed by security='none') \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*

You can control this feature from your bean configuration code

@Configuration
@EnableWebSecurity(debug = true)
public class SecurityConfig extends WebSecurityConfigurerAdapter

Still, you may want to control it from application.properties files for some profiles. Like I want to keep it on for dev only.

org.springframework.security.config.annotation.web.builders.WebSecurity.debugEnabled=true

It’s not mature

When it comes to new innovating technologies which challenge the existing paradigm, often start with something small. It takes a significant amount of development, reiterations and changes to make it compatible with existing infrastructure and platforms.

ORM (Object relational mapping) tools started a long time back. But it took years to reach a level where you can simply relay on ORM to do everything with the database. In initial year if you used ORM, you still had to use native SQL to do lot of work. Sometime simply the dialect or driver wasn’t available or compatible.

Not Suited for your use case

We are in a world that changes at an exponential rate. Whenever we see something fancy, we just want to play with it, try it out. In this love, we just use it with our app and later realize that we are using a very small part of it.

Blockchain, one of the great technologies of our time. I have seen people using it for trivial use cases like Content Management System_. Such a waste of technology, right?_

Real-life failure: I was working on a project where a portal framework was in use. It used to take a minimum of 4GB of RAM to run the container itself, not to consider hardware requirements for development. where you need to run IDE, Database and other tools. After a year of work, we found ourselves using only 5% of the features. The framework had great features but was not needed for us!

Not adding any business value

Every business uses technology. You cannot imagine a business without technology. Still, there is one thing that distinguishes them

Technology is core of your business. (Examples: Software Development, Innovation and R&D companies)
Business using technology to support/improve business. (Examples: Manufacturing, Business Consulting, Real estate etc.)

In these two categories, we can definitely say that the first ones whose core function is technology/IT innovation need to stay on the top and needs to do a lot of research and innovations. Like FB, Google, IT servicing companies, Companies developing tools for the market.

In other ones, the core function is not technology. Business value is derived from core functions, technology works as support. In these scenarios keeping the tools/applications up to date may sound like an ideal approach but doesn’t add much to business. Sometimes these upgrades even cost way more than value added.

Use it if its useful for you not because everyone else doing it

People

You need people to create and maintain a application. We all know rare techs face challenges finding right talent for it. Better not end-up in a situation, what recruitment industry call it Resource Crunch.

Learning from everyone else is great, don’t follow them!

These are some thoughts from my experience. Hope it help you make a informed decision.

You need to serialize an object that has some static properties. The answer is, you need to control the process of serialization and De-Serialization for those objects. To do this in java you need to implement the interface Externalizable.  This is similar to the Serializable interface but with more control over the process of the serialization.

java.io.Externalizable interface provides two methods writeExternal() and readExternal(). These methods are invoked whenever an object is being serialized or De-serialized. Syntax of both are shown below.

void writeExternal(ObjectOutput out) throws IOException;

void readExternal(ObjectInput in) throws IOException, ClassNotFoundException;

For demonstration, I have created a class ExternalizableEmployee which has only two Strong fields, name and a static field company.

Approach

As we know that serialization will not save a property marked as static. we can create a proxy object that has the same properties but none of them is marked as static. So when you save the proxy object it will save all the properties and vice-versa when reading. In this example, I created a static Employee class with the ExternalizableEmployee class. Within because we do not need this outside the scope of E_xternalizableEmployee_ class_._

E_xternalizableEmployee _class

import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;
import java.io.Serializable;

public class ExternalizableEmployee implements Externalizable {

	private String name;
	private static String company;

	// Inner Proxy class for Serialization.
	private static class Employee implements Serializable {
		private static final long serialVersionUID = 610993458650534361L;

		String name;
		String company;

		public Employee(String name, String company) {
			super();
			this.name = name;
			this.company = company;
		}

		public String getName() {
			return name;
		}

		public String getCompany() {
			return company;
		}

	}

	@Override
	public void writeExternal(ObjectOutput out) throws IOException {
		/*
		 * Write the object of proxy class Employee with the values of
		 * ExternalizableEmployee
		 */
		Employee emp = new Employee(name, company);
		out.writeObject(emp);
	}

	@Override
	public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
		/* Read the proxy object and initialize ExternalizableEmployee */
		Employee employee = (Employee) in.readObject();
		this.name = employee.getName();
		company = employee.getCompany();
	}

	public String getName() {
		return name;
	}

	public void setName(String name) {
		this.name = name;
	}

	public static String getCompany() {
		return company;
	}

	public static void setCompany(String company) {
		ExternalizableEmployee.company = company;
	}

}

Write ExternalizableEmployee object to File

import java.io.FileOutputStream;
import java.io.IOException;
import java.io.ObjectOutputStream;

public class WriteExternalizableEmployeeTest {
	public static void main(String[] args) {
		try (ObjectOutputStream objectOutputStream = new ObjectOutputStream(new FileOutputStream("employee.data"))) {
			// create an object of ExternalizableEmployee
			ExternalizableEmployee emp=new ExternalizableEmployee();
			emp.setName("Test");
			ExternalizableEmployee.setCompany("ByteFold");
			
			// write object to file
			objectOutputStream.writeObject(emp);
		} catch (IOException e) {
			e.printStackTrace();
		}
	}
}

Read ExternalizableEmployee object

import java.io.FileInputStream;
import java.io.IOException;
import java.io.ObjectInputStream;

public class ReadExternalizableEmployeeTest {
	public static void main(String[] args) {
		try (ObjectInputStream objectOutputStream = new ObjectInputStream(new FileInputStream("employee.data"))) {
			// Read an object of ExternalizableEmployee
			ExternalizableEmployee employee = (ExternalizableEmployee) objectOutputStream.readObject();

			System.out.println(employee.getName());
			System.out.println(employee.getCompany());
		} catch (IOException | ClassNotFoundException e) {
			e.printStackTrace();
		}
	}
}

Find the complete code at GitHub

More about Serialization

I used this framework with couple of project and now it's my first go for authentication and authorization mechanism around any application, even over Spring Security. A lot of may like Spring Security because it comes with your spring and lot of community support and documentations.

You need to create a realm that provides all the logic of Authenticating a User and Authorizing it for any access. Below is a simple realm class. (Not doing any verification, just for demonstration)

Shiro Dependency

<!-- shiro dependency -->
		<dependency>
			<groupId>org.apache.shiro</groupId>
			<artifactId>shiro-all</artifactId>
			<version>1.1.0</version>
		</dependency>

Realm

import org.apache.shiro.authc.AuthenticationException;
import org.apache.shiro.authc.AuthenticationInfo;
import org.apache.shiro.authc.AuthenticationToken;
import org.apache.shiro.authc.SimpleAuthenticationInfo;
import org.apache.shiro.authc.UsernamePasswordToken;
import org.apache.shiro.authc.credential.CredentialsMatcher;
import org.apache.shiro.authz.AuthorizationInfo;
import org.apache.shiro.authz.SimpleAuthorizationInfo;
import org.apache.shiro.realm.AuthorizingRealm;
import org.apache.shiro.subject.PrincipalCollection;

public class MyRealm extends AuthorizingRealm {

	public MyRealm() {
		super();
		setCredentialsMatcher(new CredentialsMatcher() {
			
			@Override
			public boolean doCredentialsMatch(AuthenticationToken arg0,
					AuthenticationInfo arg1) {
				System.out
						.println("MyRealm.MyRealm().new CredentialsMatcher() {...}.doCredentialsMatch()");
				return true;
			}
		});
		
	}
	
	@Override
	protected AuthorizationInfo doGetAuthorizationInfo(PrincipalCollection arg0) {
		System.out.println("MyRealm.doGetAuthorizationInfo()");
		AuthorizationInfo info=new SimpleAuthorizationInfo();
		return info;
	}

	

	@Override
	protected AuthenticationInfo doGetAuthenticationInfo(
			AuthenticationToken arg0) throws AuthenticationException {
		System.out.println("MyRealm.doGetAuthenticationInfo()");
		UsernamePasswordToken token=(UsernamePasswordToken) arg0;
		
		AuthenticationInfo info=new SimpleAuthenticationInfo(1,token.getCredentials(), getName());
		return info;
	}

}

Spring Configuration for Shiro

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">


<!-- Define the realm you want to use to connect to your back-end security datasource: -->
<bean id="myRealm" class="com.realm.MyRealm"></bean>

<bean id="securityManager" class="org.apache.shiro.mgt.DefaultSecurityManager">
    <!-- Single realm app.  If you have multiple realms, use the 'realms' property instead. -->
    <property name="realm" ref="myRealm"/>
    <property name="sessionManager.sessionListeners">
    	<list>
    	<ref bean="mySessionListener" />
    	</list>
    </property>
</bean>
<bean id="lifecycleBeanPostProcessor" class="org.apache.shiro.spring.LifecycleBeanPostProcessor"/>

<!-- For simplest integration, so that all SecurityUtils.* methods work in all cases, -->
<!-- make the securityManager bean a static singleton.  DO NOT do this in web         -->
<!-- applications - see the 'Web Applications' section below instead.                 -->
<bean class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
    <property name="staticMethod" value="org.apache.shiro.SecurityUtils.setSecurityManager"/>
    <property name="arguments" ref="securityManager"/>
</bean>

<bean id="mySessionListener" class="com.listner.MySessionListener" ></bean>
</beans>

Demo

public class ShiroTest {
	public static void main(String[] args) {
		AbstractApplicationContext context=new ClassPathXmlApplicationContext("spring.xml");
		context.registerShutdownHook();
		org.apache.shiro.subject.Subject subject=SecurityUtils.getSubject();
		AuthenticationToken token=new UsernamePasswordToken("username", "password");
		System.out.println("Login a user--");
 		subject.login(token);
 		System.out.println("User logged in---"); subject.logout(); System.out.println("User logged out"); 
    } 
}

A fully functional demo available on GitHub https://github.com/ankitkatiyar91/java-framework-examples/tree/master/spring-examples/SpringShiro

Check CMS application that usages Shiro for security https://github.com/ankitkatiyar91/cms-java

I stuck with a problem where my production team has blocked access to production URLs of individual nodes. Now I cannot check if a particular node in my cluster is working or not. After a lot of efforts, they gave us a bastion server  (a server machine that is allowed with special rights, more) that allows us access to these nodes.

Basically, my application URL is www.myurl.com that runs 10 nodes on 8080 port. Then I am allowed to access www.myurl.com URL over internet/VPN but not allowed to check individual nodes like www.node1.myurl.com:8080 etc. Now I have to use the new bastion server and log in through SSH then use dynamic port forward on that server. After that use SOCKS proxy of my browser to access these URLs.

Although, I am able to access these URLs from my browser but what about my JAVA utility that was doing all the verification without me doing all the manual work? So juggled around couple of libraries to see if I can still use that utitily and do all this.

There are couple of libraries but I explored two of them

• JSCH (Doesn’t support dynamic port forward as of now)
• Apache MINA

JSCH Example

Dependencies

<dependency>
  <groupId>com.jcraft</groupId>
  <artifactId>jsch</artifactId>
  <version>0.1.55</version>
</dependency>

Example

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;

import com.jcraft.jsch.JSch;
import com.jcraft.jsch.Session;
import com.jcraft.jsch.UserInfo;

public class TunnelJSCH {
	public static void main(String[] args) {
		TunnelJSCH t = new TunnelJSCH();
		try {
			t.go();
		} catch (Exception ex) {
			ex.printStackTrace();
		}
	}

	public void go() throws Exception {
		String host = "bastion-server-host";
		String user = "user";
		String password = "password";
		int port = 22;

		int tunnelLocalPort = 8000;
		String tunnelRemoteHost = "remote-host";
		int tunnelRemotePort = 9080;

		JSch jsch = new JSch();
		Session session = jsch.getSession(user, host, port);
		session.setPassword(password);
		localUserInfo lui = new localUserInfo();
		session.setUserInfo(lui);
		System.out.println("Creating connection to server:" + host);
		session.connect();
		System.out.println("Connected to server, initializing Port forwarding ");
		session.setPortForwardingL(tunnelLocalPort, tunnelRemoteHost, tunnelRemotePort);
		
		System.out.println("Port forward successful forwarding: localhost:" + tunnelLocalPort + " -> "
				+ tunnelRemoteHost + ":" + tunnelRemotePort);

		try {

			URL url = new URL("http://localhost:8000");
			System.out.println("Reading data from URL: " + url);
			BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
			System.out.println("================== Data From URL ==================\n");
			String inputLine;
			while ((inputLine = in.readLine()) != null)
				System.out.println(inputLine);
			in.close();

			System.out.println("================== Data From URL ==================\n");
		} catch (Exception e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

		if (session != null && session.isConnected()) {
			System.out.println("Closing SSH Connection");
			session.disconnect();
		}

	}

	class localUserInfo implements UserInfo {
		String passwd;

		public String getPassword() {
			return passwd;
		}

		public boolean promptYesNo(String str) {
			return true;
		}

		public String getPassphrase() {
			return null;
		}

		public boolean promptPassphrase(String message) {
			return true;
		}

		public boolean promptPassword(String message) {
			return true;
		}

		public void showMessage(String message) {
		}
	}
}

Apache MINA

Dependencies

<!-- https://mvnrepository.com/artifact/org.apache.mina/mina-core -->
		<dependency>
			<groupId>org.apache.mina</groupId>
			<artifactId>mina-core</artifactId>
			<version>3.0.0-M2</version>
		</dependency>

		<!-- https://mvnrepository.com/artifact/org.apache.sshd/sshd-core -->
		<dependency>
			<groupId>org.apache.sshd</groupId>
			<artifactId>sshd-core</artifactId>
			<version>2.1.0</version>
		</dependency>

		<dependency>
			<groupId>org.apache.sshd</groupId>
			<artifactId>sshd-putty</artifactId>
			<version>2.1.0</version>
		</dependency>

		<dependency>
			<groupId>org.apache.sshd</groupId>
			<artifactId>sshd-common</artifactId>
			<version>2.1.0</version>
		</dependency>

Example

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.InetSocketAddress;
import java.net.MalformedURLException;
import java.net.Proxy;
import java.net.URISyntaxException;
import java.net.URL;
import java.nio.file.Paths;
import java.security.GeneralSecurityException;
import java.security.KeyPair;
import java.util.Collection;
import java.util.Collections;
import java.util.List;

import org.apache.sshd.client.SshClient;
import org.apache.sshd.client.auth.hostbased.HostKeyIdentityProvider;
import org.apache.sshd.client.keyverifier.AcceptAllServerKeyVerifier;
import org.apache.sshd.client.session.ClientSession;
import org.apache.sshd.common.NamedFactory;
import org.apache.sshd.common.config.keys.loader.pem.PEMResourceParserUtils;
import org.apache.sshd.common.config.keys.loader.putty.PuttyKeyUtils;
import org.apache.sshd.common.forward.PortForwardingEventListener;
import org.apache.sshd.common.session.Session;
import org.apache.sshd.common.util.net.SshdSocketAddress;
import org.apache.sshd.server.channel.PuttyRequestHandler;
import org.apache.sshd.server.forward.AcceptAllForwardingFilter;
import org.apache.sshd.server.keyprovider.SimpleGeneratorHostKeyProvider;

/**
 * This class
 * 
 * @author Ankit Katiyar
 *
 */
public class AmazonTest {

	private static String BASTION_SERVER_PASSWORD = "P@ssword1";
	private static final String BASTION_SERVER_USER = "ec2-user";
	private static final String BASTION_SEREVR_HOST = "ec2-18-191-207-91.us-east-2.compute.amazonaws.com";
	private static final String URL_TO_ACCESS = "http://www.google.com";

	public static void main(String[] args) {

		try {
			
			
			Collection<KeyPair> keys = null;
			// OPtional loading keys from a PEM file
			//keys=PEMResourceParserUtils.getPEMResourceParserByAlgorithm("RSA").loadKeyPairs(ClassLoader.getSystemResource("local-ps-test.pem").toURI().toURL(), null);
			
			// Optional: Using Putty key for login 
			 keys=PuttyKeyUtils.DEFAULT_INSTANCE.loadKeyPairs(ClassLoader.getSystemResource("local-ps-private-key.ppk").toURI().toURL(), null);
			 
			SshClient client = SshClient.setUpDefaultClient();
			client.setForwardingFilter(AcceptAllForwardingFilter.INSTANCE);
			client.setServerKeyVerifier(AcceptAllServerKeyVerifier.INSTANCE);
			client.start();

			// using the client for multiple sessions...
			try (ClientSession session = client.connect(BASTION_SERVER_USER, BASTION_SEREVR_HOST, 22).verify()
					.getSession()) {
				// IF you use password to login provide here
				// session.addPasswordIdentity(BASTION_SERVER_PASSWORD); // for password-based
				// authentication
				
				session.addPublicKeyIdentity(keys.iterator().next());

				// authentication
				// Note: can add BOTH password AND public key identities - depends on the
				// client/server security setup

				session.auth().verify(10000);
				// start using the session to run commands, do SCP/SFTP, create local/remote
				// port forwarding, etc...

				session.addPortForwardingEventListener(new PortForwardingEventListener() {

					@Override
					public void establishedDynamicTunnel(Session session, SshdSocketAddress local,
							SshdSocketAddress boundAddress, Throwable reason) throws IOException {
						// TODO Auto-generated method stub
						PortForwardingEventListener.super.establishedDynamicTunnel(session, local, boundAddress, reason);

						System.out.println("Dynamic Forword Tunnel is Ready");
					}
				});

				SshdSocketAddress sshdSocketAddress = session
						.startDynamicPortForwarding(new SshdSocketAddress("localhost", 8000));

				System.out.println("Host: " + sshdSocketAddress.getHostName());
				System.out.println("Port: " + sshdSocketAddress.getPort());

				// Create a Proxy object to work with
				Proxy proxy = new Proxy(Proxy.Type.SOCKS,
						new InetSocketAddress(sshdSocketAddress.getHostName(), sshdSocketAddress.getPort()));

				/**
				 * Now you can use this proxy instance into any URL until this SSH session is active. 
				 */
				
				// TEST one URL
				HttpURLConnection connection = (HttpURLConnection) new URL(URL_TO_ACCESS).openConnection(proxy);

				System.out.println("Proxy work:" + connection.getURL());

				BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
				System.out.println("================== Data From URL ==================\n");
				String inputLine;
				while ((inputLine = in.readLine()) != null)
					System.out.println(inputLine);
				in.close();

				System.out.println("================== Data From URL ==================\n");
			} catch (IOException e1) {
				// TODO Auto-generated catch block
				e1.printStackTrace();
			} catch (Exception e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}
		} catch (Exception e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}

	}

}

View complete project at GitHub 

https://github.com/ankitkatiyar91/java-framework-examples/tree/master/java-tunneling

After some googling I found that JAVA supports JVM shutdown hooks. These hooks are some threads that are executed when JVM is shutting down. Java’s Runtime class provide an addShutdownHook(Thread hook) method. This method will add this hook  thread to be executed when all the tasks are done and JVM this about to shutdown/close.

public class JVMShutdownHook
{

    public static void main(String[] args)
    {
        System.out.println("Main start... ");
        Runtime.getRuntime().addShutdownHook(new Thread()
        {
            @Override
            public void run()
            {
                System.out.println("Finally shut down hook called");
            }
        });

        System.out.println("Main end.... ");
    }

}

Output

Main start... 
Main end....   
Finally shut down hook called

You can see from the output that code inside hook is executed at the end. This way you can handle events, free resources or something else like updating some flags/logs/process file when you application is shutting down. We often need these while running some jobs on server.

A moment’s insight is sometimes worth a life’s experience.
–Oliver Wendell Holmes (1809-1894)

it will give you an exception as java.lang.ArrayIndexOutOfBoundsException.

Here I have one implementation of a binary search where I will not use the length property of the array.

Example

import static java.lang.Math.pow;

public class BinarySearchWithoutLength {

  public static Integer binarySearch(final Integer[] elements, final Integer find) {
    /*
     * Start looking for element at the 0th position and move in multiple of 2. like 2^0, 2^1, 2^2,
     * 2^3
     */
    Integer index = 0;
    Integer exp = 0; // initial exponent value will be 0
    try {
      while (true) {
        // check if we found the element at 2^exp index
        if (elements[index] == find) {
          return index;
        }
        if (elements[index] < find) {
          index = (int) pow(2, exp);
          exp += 1;
        } else {
          break;
        }
      }
    } catch (ArrayIndexOutOfBoundsException e) {
      /*
       * ignore this exception as we expect it to happen if we go beyond the size of the array
       * without finding element at position 2^exp
       */
    }
    int mid; // it will represent mid value where lookup will happen
    int left = (index / 2) + 1;
    int right = index - 1;
    /*
     * we will start our lookup from the middle of the array and then shift towards other half of
     * the array.
     */
    while (left <= right) {
      mid = ((right + left) / 2);
      try {
        if (elements[mid] == find) {
          /*
           * As we have found one matching element. Stop
           */
          return mid;
        } else {
          if (elements[mid] > find) {
            /* whenever you find and element which is greater than the find */
            right = mid - 1;
          } else {
            left = mid + 1;
          }
        }
      } catch (ArrayIndexOutOfBoundsException e) {
        /*
         * we got this because we reached beyond the size, shift our right index to one place left
         * of the mid
         */
        right = mid - 1;
      }
    }
    return -1;
  }

  public static void main(String[] args) {
    Integer[] sample_1 = new Integer[] {-3, -2, 3, 4, 7};
    Integer[] sample_2 = new Integer[] {-3, -2, -1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13};
    Integer[] sample_3 = new Integer[] {1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13};
    System.out.println("Index of 4: " + binarySearch(sample_1, 4));
    System.out.println("Index of 4: " + binarySearch(sample_2, 4));
    System.out.println("Index of 4: " + binarySearch(sample_3, 10));
    System.out.println("Index of 4: " + binarySearch(sample_3, 0));
  }
}

Download from GitHub