Zoo-keeper

ZK Motto

the motto “ZooKeeper: Because Coordinating Distributed Systems is a Zoo.”

Features of Zookeeper

  • Synchronization − Mutual exclusion and co-operation between server processes.
  • Ordered Messages - The strict ordering means that sophisticated synchronization primitives can be implemented at the client.
  • Reliability - The reliability aspects keep it from being a single point of failure.
  • Atomicity − Data transfer either succeeds or fails completely, but no transaction is partial.
  • High performant - The performance aspects of Zookeeper means it can be used in large, distributed systems.
  • Distributed.
  • High avaliablity.
  • Fault-tolerant.
  • Loose coupling.
  • Partial failure.
  • High throughput and low latency - data is stored data in memory and on disk as well.
  • Replicated.
  • Automatic failover: When a Zookeeper dies, the session is automatically migrated over to another Zookeeper.

Different type of data

When designing an application with ZooKeeper, one ideally separates application data from control or coordination data. For example, the users of a web-mail service are interested in their mailbox content, but not on which server is handling the requests of a particular mailbox. The mailbox content is application data, whereas the mapping of the mailbox to a specific mail server is part of the coordination data (or metadata). A ZooKeeper ensemble manages the latter.

The multiple processes consequently need to implement mutual exclusion. We can actually think of the task of acquiring mastership as the one of acquiring a lock: the process that acquires the mastership lock exercises the role of master.

Coordination does not always take the form of synchronization primitives like leader election or locks. Configuration metadata is often used as a way for a process to convey what others should be doing. For example, in a master-worker system, workers need to know the tasks that have been assigned to them, and this information must be available even if the master crashes.

How world work without ZooKeeper

It is certainly possible to build distributed systems without using ZooKeeper. ZooKeep‐ er, however, offers developers the possibility of focusing more on application logic rather than on arcane distributed systems concepts. Programming distributed systems without ZooKeeper is possible, but more difficult.

What does ZooKeeper does not do

The ensemble of ZooKeeper servers manages critical application data related to coor‐ dination. ZooKeeper is not for bulk storage. For bulk storage of application data, there are a number of options available, such as databases and distributed file systems. When designing an application with ZooKeeper, one ideally separates application data from control or coordination data.

ZooKeeper, however, does not implement the tasks for you. It does not elect a master or track live processes for the application out of the box. Instead, it provides the tools for implementing such tasks. The developer decides what coordination tasks to implement.

Processes in a distributed system have two broad options for communication: they can exchange messages directly through a network, or read and write to some shared storage. ZooKeeper uses the shared storage model to let applications implement coordination and synchronization primitives. But shared storage itself requires network communi‐ cation between the processes and the storage. It is important to stress the role of network communication because it is an important source of complications in the design of a distributed system.

This scenario leads to a problem commonly called split-brain: two or more parts of the system make progress independ‐ ently, leading to inconsistent behavior. As part of coming up with a way to cope with master failures, it is critical that we avoid split-brain scenarios.

Tasks

The following requirements for our master-worker architecture: Master election It is critical for progress to have a master available to assign tasks to workers. Crash detection The master must be able to detect when workers crash or disconnect. Group membership management The master must be able to figure out which workers are available to execute tasks. Metadata management The master and the workers must be able to store assignments and execution sta‐ tuses in a reliable manner.

CAP

known as CAP, which stands for Consistency, Availability, and Partition-tolerance, says that when designing a distributed system we may want all three of those properties, but that no system can handle all three.2 Zoo‐ Keeper has been designed with mostly consistency and availability in mind, although it also provides read-only capability in the presence of network partitions.

ZooKeeper Basics

Several primitives used for coordination are commonly shared across many applica‐ tions. Consequently, one way of designing a service used for coordination is to come up with a list of primitives, expose calls to create instances of each primitive, and ma‐ nipulate these instances directly. For example, we could say that distributed locks con‐ stitute an important primitive and expose calls to create, acquire, and release locks. Such a design, however, suffers from a couple of important shortcomings. First, we need to either come up with an exhaustive list of primitives used beforehand, or keep ex‐ tending the API to introduce new primitives. Second, it does not give flexibility to the application using the service to implement primitives in the way that is most suitable for it. We consequently have taken a different path with ZooKeeper. ZooKeeper does not ex‐ pose primitives directly. Instead, it exposes a file system-like API comprised of a small set of calls that enables applications to implement their own primitives. We typically use recipes to denote these implementations of primitives. Recipes include ZooKeeper operations that manipulate small data nodes, called znodes, that are organized hier‐ archically as a tree, just like in a file system.

znodes

a few other znodes that could be useful in a master- worker configuration:

  • The /workers znode is the parent znode to all znodes representing a worker avail‐ able in the system. Figure 2-1 shows that one worker (foo.com:2181) is available. If a worker becomes unavailable, its znode should be removed from /workers.
  • The /tasks znode is the parent of all tasks created and waiting for workers to execute them. Clients of the master-worker application add new znodes as children of /tasks to represent new tasks and wait for znodes representing the status of the task.
  • The /assign znode is the parent of all znodes representing an assignment of a task to a worker. When a master assigns a task to a worker, it adds a child znode to /assign.

    API

    API Overview Znodes may or may not contain data. If a znode contains any data, the data is stored as a byte array. The exact format of the byte array is specific to each application, and ZooKeeper does not directly provide support to parse it. Serialization packages such as Protocol Buffers, Thrift, Avro, and MessagePack may be handy for dealing with the format of the data stored in znodes, but sometimes string encodings such as UTF-8 or ASCII suffice.

The ZooKeeper API exposes the following operations:

create /path data
Creates a znode named with /path and containing data
delete /path
Deletes the znode /path
exists /path
Checks whether /path exists
setData /path data
Sets the data of znode /path to data
getData /path
Returns the data in /path
getChildren /path
Returns the list of children under /path
One important note is that ZooKeeper does not allow partial writes or reads of the znode data. When setting the data of a znode or reading it, the content of the znode is replaced or read entirely.

ZooKeeper clients connect to a ZooKeeper service and establish a session through which they make API calls.

If a worker becomes unavailable, its session expires and its znode in /workers disappears automatically.

An ephemeral znode can be deleted in two situations:

  1. When the session of the client creator ends, either by expiration or because it ex‐ plicitly closed.
  2. When a client, not necessarily the creator, deletes it.

Sequential znodes

A znode can also be set to be sequential. A sequential znode is assigned a unique, mo‐ notonically increasing integer. This sequence number is appended to the path used to create the znode. For example, if a client creates a sequential znode with the path /tasks/ task-, ZooKeeper assigns a sequence number, say 1, and appends it to the path. The path of the znode becomes /tasks/task-1. Sequential znodes provide an easy way to create znodes with unique names. They also provide a way to easily see the creation order of znodes.

watch

This is a common problem with polling. To replace the client polling, we have opted for a mechanism based on notifications: clients register with ZooKeeper to receive notifi‐ cations of changes to znodes. Registering to receive a notification for a given znode consists of setting a watch. A watch is a one-shot operation, which means that it triggers one notification. To receive multiple notifications over time, the client must set a new watch upon receiving each notification.

Versions

Each znode has a version number associated with it that is incremented every time its data changes. A couple of operations in the API can be executed conditionally: setDa ta and delete. Both calls take a version as an input parameter, and the operation suc‐ ceeds only if the version passed by the client matches the current version on the server. The use of versions is important when multiple ZooKeeper clients might be trying to perform operations over the same znode. For example, suppose that a client c1 writes a znode /config containing some configuration. If another client c2 concurrently updates the znode, the version c1 has is stale and the setData of c1 must not succeed. Using versions avoids such situations. In this case, the version that c1 uses when writing back doesn’t match and the operation fails.

ZooKeeper Architecture

Now that we have discussed at a high level the operations that ZooKeeper exposes to applications, we need to understand more of how the service actually works. Applica‐ tions make calls to ZooKeeper through a client library. The client library is responsible for the interaction with ZooKeeper servers.

ZooKeeper servers run in two modes: standalone and quorum. Standalone mode is pretty much what the term says: there is a single server, and ZooKeeper state is not replicated. In quorum mode, a group of ZooKeeper servers, which we call a ZooKeeper ensemble, replicates the state, and together they serve client requests. From this point on, we use the term “ZooKeeper ensemble” to denote an installation of servers. This installation could contain a single server and operate in standalone mode or contain a group of servers and operate in quorum mode.

ZooKeeper Quorums

In quorum mode, ZooKeeper replicates its data tree across all servers in the ensemble. But if a client had to wait for every server to store its data before continuing, the delays might be unacceptable. In public administration, a quorum is the minimum number of legislators required to be present for a vote. In ZooKeeper, it is the minimum number of servers that have to be running and available in order for ZooKeeper to work. This number is also the minimum number of servers that have to store a client’s data before telling the client it is safely stored. For instance, we might have five ZooKeeper servers in total, but a quorum of three. So long as any three servers have stored the data, the client can continue, and the other two servers will eventually catch up and store the data.

It is important to choose an adequate size for the quorum. Quorums must guarantee that, regardless of delays and crashes in the system, any update request the service pos‐ itively acknowledges will persist until another request supersedes it.

The bottom line is that we should always shoot for an odd number of servers.

sessions

All operations a client submits to ZooKeeper are associated to a session. When a session ends for any reason, the ephemeral nodes created during that session disappear.

the session may be moved to a different server if the client has not heard from its current server for some time. Moving a session to a different server is handled transparently by the ZooKeeper client library.

Sessions offer order guarantees, which means that requests in a session are executed in FIFO (first in, first out) order. Typically, a client has only a single session open, so its requests are all executed in FIFO order. If a client has multiple concurrent sessions, FIFO ordering is not necessarily preserved across the sessions.

Commands

[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 5] stat /master
cZxid = 0x4
ctime = Mon Aug 20 21:10:23 AEST 2018
mZxid = 0x4
mtime = Mon Aug 20 21:10:23 AEST 2018
pZxid = 0x4
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x10003c70e250001
dataLength = 11
numChildren = 0


[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 8] create /workers ""
Created /workers
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 9] create /tasks ""
Created /tasks
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 10] create /assign ""
Created /assign
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 11] ls /
[assign, master, tasks, workers, zookeeper]

In a real application, these znodes need to be created either by a primary process before it starts assigning tasks or by some bootstrap procedure. Regardless of how they are created, once they exist, the master needs to watch for changes in the children of /workers and /tasks: [zk: localhost:2181(CONNECTED) 4] ls /workers true [] [zk: localhost:2181(CONNECTED) 5] ls /tasks true [] [zk: localhost:2181(CONNECTED) 6] Note that we have used the optional true parameter with ls, as we did before with stat on the master. The true parameter, in this case, creates a watch for changes to the set of children of the corresponding znode.

[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 14] create -e /workers/todd-worker1 ""
Created /workers/todd-worker1

WATCHER::

WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/workers
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 15]

Recall that the master has set a watch for changes to the children of /workers. Once the worker creates a znode under /workers, the master observes the following notification: WATCHER:: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/workers

Tasks workflows

  • Clients add tasks to the system. Here we assume that the client asks the master-worker system to run a command cmd. To add a task to the system, a client executes the following: [zk: localhost:2181(CONNECTED) 0] create -s /tasks/task- “cmd” Created /tasks/task-0000000000
  • The client now has to wait until the task is executed.
  • The worker that executes the task creates a status znode for the task once the task completes.
  • The client determines that the task has been executed when it sees that a status znode for the task has been created;
  • the client consequently must watch for the creation of the status znode:
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 18] create -s /tasks/task- "cmd"
Created /tasks/task-0000000000
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 19] ls /tasks
[task-0000000000]
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 20] ls -w /tasks/task-0000000000 
[]
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 21] ls -w /workers
[todd-worker1]
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 22] create /assign/todd-worker1/task-0000000000 ""
Ephemerals cannot have children: /assign/todd-worker1/task-0000000000
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 23] delete /assign/todd-worker1
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 24] create /assign/todd-worker1
Created /assign/todd-worker1
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 25] create /assign/todd-worker1/task-0000000000 ""
Created /assign/todd-worker1/task-0000000000
[zk: 127.0.0.1:2181,127.0.0.1:2182(CONNECTED) 26] ls /assign/todd-worker1
[task-0000000000]



# Once the worker finishes executing the task, it adds a status znode to /tasks:
    [zk: localhost:2181(CONNECTED) 4] create /tasks/task-0000000000/status "done"
    Created /tasks/task-0000000000/status
    [zk: localhost:2181(CONNECTED) 5]
# and the client receives a notification and checks the result:
WATCHER::
    WatchedEvent state:SyncConnected type:NodeChildrenChanged
    path:/tasks/task-0000000000
    [zk: localhost:2181(CONNECTED) 2] get /tasks/task-0000000000
    "cmd"

ZooKeeper API

Setting the ZooKeeper CLASSPATH

ZOOBINDIR=”/bin" . "$ZOOBINDIR"/zkEnv.sh

handle

The ZooKeeper API is built around a ZooKeeper handle that is passed to every API call. This handle represents a session with ZooKeeper. A session that is established with one ZooKeeper server will migrate to another ZooKeeper server if its connection is broken. As long as the session is alive, the handle will remain valid, and the ZooKeeper client library will continually try to keep an active connection to a ZooKeeper server to keep the session alive. If the handle is closed, the ZooKeeper client library will tell the ZooKeeper servers to kill the session. If ZooKeeper decides that a client has died, it will invalidate the session. If a client later tries to reconnect to a Zoo‐ Keeper server using the handle that corresponds to the invalidated session, the Zoo‐ Keeper server informs the client library that the session is no longer valid and the handle returns errors for all operations.

The constructor that creates a ZooKeeper handle usually looks like: ZooKeeper( String connectString, int sessionTimeout, Watcher watcher)

Implementing a Watcher

To receive notifications from ZooKeeper, we need to implement watchers. Let’s look a bit more closely at the Watcher interface. It has the following declaration: public interface Watcher { void process(WatchedEvent event); }

Sample ZooKeeper handle

import org.apache.zookeeper.ZooKeeper; 
import org.apache.zookeeper.Watcher;

public class Master implements Watcher { 
  ZooKeeper zk;
        String hostPort;

Master(String hostPort) 
{ this.hostPort = hostPort;
}
void startZK() {
zk = new ZooKeeper(hostPort, 15000, this);
}
public void process(WatchedEvent e) { System.out.println(e);
}
public static void main(String args[]) throws Exception {
Master m = new Master(args[0]);
            m.startZK();
            // wait for a bit
            Thread.sleep(60000);
        }
}

nce we have connected to ZooKeeper, there will be a background thread that will maintain the ZooKeeper session. This thread is a daemon thread, which means that the program may exit even if the thread is still active. Here we sleep for a bit so that we can see some events come in before the program exits. We can compile this simple example using the following: $ javac -cp $CLASSPATH Master.java Once we have compiled Master.java, we run it and see the following: $ java -cp $CLASSPATH Master 127.0.0.1:2181

disconnect

When developers see the Disconnected event, some think they need to create a new ZooKeeper handle to reconnect to the service. Do not do that! See what happens when you start the server, start the Master, and then stop and start the server while the Master is still running. You should see the SyncConnected event followed by the Disconnec ted event and then another SyncConnected event. The ZooKeeper client library takes care of reconnecting to the service for you. Unfortunately, network outages and server failures happen. Usually, ZooKeeper can deal with these failures.

2024

3 minute read

awesome mr W

2 minute read

You are not a drop in the ocean, you are the entire ocean in a drop.

Flyway Self Healing

less than 1 minute read

how to let your flyway database scheme migrate more robustly and self healing

Flyway Self Healing

less than 1 minute read

how to let your flyway database scheme migrate more robustly and self healing

Back to top ↑

2023

Az Cli

less than 1 minute read

To run commands in VMs in Azure

Cheap and flexible computing

1 minute read

whether it seems possible or not - go for it Cheaper X 2 to EC2, to use Fargate Spot With Fargate Spot you can run interruption tolerant Amazon ECS t...

Understanding React export a Component

1 minute read

A young idler, an old beggar. - William Shakespeare Understanding React export a Component In this blog post, we will dive into the code of the RepoU...

UUID deep dive

4 minute read

A young idler, an old beggar. - William Shakespeare

which-port-my-service-is-running

2 minute read

Summary As a Java developer, it’s important to know how to find out which port number your Spring service is running on. This information is useful when you ...

Elk Search Tips

less than 1 minute read

message:/'Invoking SP with quoteContext*werqewr-1234asdf-sdf23-9d83-asdf23*'/

what is StrictHostKeyChecking in ssh

less than 1 minute read

What’s and how to avoid error of the authenticity of host ‘xxx’ can’t be established You can suppress the “The authenticity of host ‘’ can’t be established” ...

Chinese Verb

less than 1 minute read

知其雄，守其雌 什么意思

What’s Difference Of Cny And Cnh

3 minute read

what’s difference of CNY and CNH CNY and CNH are both currencies used in China, but they are different in a few important ways:

Spring Cloud Master Piece 9

7 minute read

What’s spring cloud config Spring Cloud Config is a distributed configuration server that provides a centralized location to manage external properties for a...

Spring Cloud Master Piece 6

1 minute read

Sample me build a micro service payment system with spring cloud Here’s an example of building a microservice payment system using Spring Cloud:

Spring Cloud Master Piece 2

5 minute read

what’s usage of bootstrap yml In a Spring Boot application, the bootstrap.yml (or bootstrap.properties) file is used for configuring the application’s enviro...

Spring Cloud Master Piece 1

7 minute read

what’s API gateway An API Gateway is a key component in microservices architecture that acts as a single entry point for client requests to a microservices-b...

Master Microfrontends

4 minute read

“The only way to do great work is to love what you do.” - Steve Jobs

How To Config JFR Java Flight Control

2 minute read

“Climb the mountains and get their good tidings. Nature’s peace will flow into you as sunshine flows into trees. The winds will blow their own freshness i...

Google マップ内の写真のコメントが表示されない

less than 1 minute read

紹介 私は、私のOppo Androidスマートフォンのアプリ「Googleマップ」で奇妙な問題が発生していることに気づきました。Googleマップで特定の場所（例えば「中央公園」）を検索すると、通常、このアプリは公園の写真やコメントリストを表示するはずです。例えば、誰かが公園の芝生や川の写真を投稿し、便利な場所...

master-cglib-in-java

2 minute read

You are not a drop in the ocean, you are the entire ocean in a drop.

Back to top ↑

2022

Minium Workable Mvp Vimrc

6 minute read

”—————————————————————- “ 4. User interface “—————————————————————- “ Set X lines to the cursor when moving vertically set scrolloff=0

Linux Tips

5 minute read

Remember, some things have to end for better things to begin.

Back to top ↑

2021

How to user fire extinguisher

less than 1 minute read

Summary As you know, staff and your safety is paramount. So what if emergency take place, such as fire in office, how to help yourself and your colleagues by...

Deep dive into Kubernetes Client API

1 minute read

Summary To talk to K8s for getting data, there are few approaches. While K8s’ official Java library is the most widely used one. This blog will look into thi...

Whitelabel Error Page

less than 1 minute read

Summary Whitelabel Error Page is the default error page in Spring Boot web app. It provide a more user-friently error page whenever there are any issues when...

Debts in a nutshell

less than 1 minute read

A debt security represents a debt owed by the issuer to an investor. Here, the investor acts as a lender to the issuer which may be a government, organisatio...

Back to top ↑

2020

How to process data from S3 download URL

less than 1 minute read

S3 download URL As you know, AWS S3 object can be downloaded/processed by S3 download URL. I’m showing you two examples on how to process S3 Object by NIO f...

Debug Stuck IntelliJ

1 minute read

What happened to a debug job hanging in IntelliJ (IDEAS) IDE? You may find when you try to debug a class in Intellij but it stuck there and never proceed, e....

Awesome Kotlin

less than 1 minute read

Difference with Scala Kotlin takes the best of Java and Scala, the response times are similar as working with Java natively, which is a considerable advantag...

JVM热身

1 minute read

此文是作者英文原文的翻译文章，英文原文在：http://todzhang.com/posts/2018-06-10-jvm-warm-up/

Mock in kotlin

2 minute read

Argument Matching & Answers For example, you have mocked DOC with call(arg: Int): Intfunction. You want to return 1 if argument is greater than 5 and -1 ...

Mock in kotlin

2 minute read

Argument Matching & Answers For example, you have mocked DOC with call(arg: Int): Intfunction. You want to return 1 if argument is greater than 5 and -1 ...

Docker

1 minute read

Dockers Concepts

Curl

less than 1 minute read

Linux Curl command

AOP

less than 1 minute read

The concept of join points as matched by pointcut expressions is central to AOP, and Spring uses the AspectJ pointcut expression language by default.

Micrometer notes

less than 1 minute read

As a general rule it should be possible to use the name as a pivot. Dimensions allow a particular named metric to be sliced to drill down and reason about th...

Awesome SSL certificates and HTTPS

16 minute read

What’s TLS TLS (Transport Layer Security) and its predecessor, SSL (Secure Sockets Layer), are security protocols designed to secure the communication betwee...

JVM warm up by Escape Analysis

13 minute read

Why JVM need warm up I don’t know how and why you get to this blog. But I know the key words in your mind are “warm” for JVM. As the name “warm up” suggested...

Java Concurrent

54 minute read

This blog is about noteworthy pivot points about Java Concurrent Framework Back to Java old days there were wait()/notify() which is error prone, while fr...

Back to top ↑

2019

Conversations with God

less than 1 minute read

Feelings is the language of the soul. If you want to know what’s true for you about something, look to how your’re feeling about.

Kafka In Spring

4 minute read

Enable Kafka listener annotated endpoints that are created under the covers by a AbstractListenerContainerFactory. To be used on Configuration classes as fol...

Mifid

1 minute read

FX Spot is not covered by the regulation, as it is not considered to be a financial instrument by ESMA, the European Union (EU) regulator. As FX is considere...

Foreign Exchange

less than 1 minute read

currency pairs Direct ccy: means USD is part of currency pair Cross ccy: means ccy wihtout USD, so except NDF, the deal will be split to legs, both with...

Back to top ↑

2018

Seconds

less than 1 minute read

nano seconds

Citrix receiver

less than 1 minute read

“Cannot connect to remote desktop” with Citrix Receiver

Guice

7 minute read

A new type of Juice Put simply, Guice alleviates the need for factories and the use of new in your Java code. Think of Guice’s @Inject as the new new. You wi...

YAML

less than 1 minute read

Key points All YAML files (regardless of their association with Ansible or not) can optionally begin with — and end with …. This is part of the YAML format a...

Mockito

less than 1 minute read

Feature

Protobuf

5 minute read

What are protocol buffers?

Sudo in a Nutshell

6 minute read

Sudo in a Nutshell Sudo (su “do”) allows a system administrator to give certain users (or groups of users) the ability to run some (or all) commands as root...

Zoo-keeper

14 minute read

ZK Motto the motto “ZooKeeper: Because Coordinating Distributed Systems is a Zoo.”

Cucumber

28 minute read

Acceptance testing vs unit test It’s sometimes said that unit tests ensure you build the thing right, whereas acceptance tests ensure you build the right thi...

Scala

less than 1 minute read

Scala String

akka framework of scala

27 minute read

philosophy The actor model adopts the philosophy that everything is an actor. This is similar to the everything is an object philosophy used by some object-o...

Apache Camel

8 minute read

Camel’s message model In Camel, there are two abstractions for modeling messages, both of which we’ll cover in this section. org.apache.camel.Message—The ...

JXM

less than 1 minute read

Exporting your beans to JMX The core class in Spring’s JMX framework is the MBeanExporter. This class is responsible for taking your Spring beans and registe...

Solace MQ

1 minute read

Solace PubSub+ It is a message broker that lets you establish event-driven interactions between applications and microservices across hybrid cloud environmen...

Apigee

4 minute read

App deployment, configuration management and orchestration - all from one system. Ansible is powerful IT automation that you can learn quickly.

Ansible

10 minute read

Ansible: What Is It Good For? Ansible is often described as a configuration management tool, and is typically mentioned in the same breath as Chef, Puppet, a...

flexbox

4 minute read

How Flexbox works — explained with big, colorful, animated gifs

KDB

18 minute read

KDB However kdb+ evaluates expressions right-to-left. There are no precedence rules. The reason commonly given for this behaviour is that it is a much simple...

Portactor

1 minute read

Better to use smart wait

Agile and SCRUM

less than 1 minute read

Key concept In Scrum, a team is cross functional, meaning everyone is needed to take a feature from idea to implementation.

rxjs pipe in depth

6 minute read

https://stormforger.com/blog/2016/07/08/types-of-performance-testing/

Strategy-Of-Openshift-Releases

less than 1 minute read

Release & Testing Strategy There are various methods for safely releasing changes to Production. Each team must select what is appropriate for their own ...

NodeJs Notes

less than 1 minute read

commands to read files var lineReader = require(‘readline’).createInterface({ input: require(‘fs’).createReadStream(‘C:\dev\node\input\git_reset_files.tx...

Minium Viable Product

less than 1 minute read

https://blog.leanstack.com/minimum-viable-product-mvp-7e280b0b9418

CORS :Cross-Origin Resource Sharing

1 minute read

Cross-Origin Request Sharing - CORS (A.K.A. Cross-Domain AJAX request) is an issue that most web developers might encounter, according to Same-Origin-Policy,...

ngrx

2 minute read

Why @Effects? In a simple ngrx/store project without ngrx/effects there is really no good place to put your async calls. Suppose a user clicks on a button or...

iOS programming

3 minute read

View A view is also a responder (UIView is a subclass of UIResponder). This means that a view is subject to user interactions, such as taps and swipes. Thus,...

Back to top ↑

2017

cloud computering

less than 1 minute read

openshift vs openstack The shoft and direct answer is `OpenShift Origin can run on top of OpenStack. They are complementary projects that work well together....

cloud computering

3 minute read

Concepts Cloud computing is the on-demand demand delivery of compute database storage applications and other IT resources through a cloud services platform v...

Redux

less than 1 minute read

whats @Effects You can almost think of your Effects as special kinds of reducer functions that are meant to be a place for you to put your async calls in suc...

reactive programing

less than 1 minute read

The second advantage to a lazy subscription is that the observable doesn’t hold onto data by default. In the previous example, each event generated by the in...

Container

2 minute read

The Docker project was responsible for popularizing container development in Linux systems. The original project defined a command and service (both named do...

promise vs observiable

less than 1 minute read

The drawback of using Promises is that they’re unable to handle data sources that produce more than one value, like mouse movements or sequences of bytes in ...

JDK source

less than 1 minute read

interface RandomAccess Marker interface used by List implementations to indicate that they support fast (generally constant time) random access. The primary ...

SSH SFTP

4 minute read

Secure FTP SFTP over FTP is the equivalant of HTTPS over HTTP, the security version

AWS Tips

less than 1 minute read

After establishing a SSH session, you can install a default web server by executing sudo yum install httpd -y. To start the web server, type sudo service htt...

Oracle

less than 1 minute read

ORA-12899: Value Too Large for Column

Spring notes

27 minute read

Spring Bean Life Cycle Callback Methods

Kindle notes

113 minute read

#《亿级流量网站架构核心技术》目录一览 TCP四层负载均衡 使用Hystrix实现隔离 基于Servlet3实现请求隔离 限流算法 令牌桶算法 漏桶算法 分布式限流 redis+lua实现 Nginx+Lua实现 使用sharding-jdbc分库分表 Disruptor+Redis...

Java Security Notes

7 minute read

Java Security well-behaved: programs should be prevent from consuming too much system resources

R Language

less than 1 minute read

s<-read.csv("C:/Users/xxx/dev/R/IRS/SHH_SCHISHG.csv") # aggregate s2<-table(s$Original.CP) s3<-as.data.frame(s2) # extract by Frequency ordered s3...

SSH and Cryptography

35 minute read

SFTP versus FTPS SS: Secure Shell An increasing number of our customers are looking to move away from standard FTP for transferring data, so we are ofte...

Eclipse notes

less than 1 minute read

How do I remove a plug-in? Run Help > About Eclipse > Installation Details, select the software you no longer want and click Uninstall. (On Macintosh i...

Java JVM

less than 1 minute read

Class loading subsystem

Maven-Notes

27 minute read

Maven philosophy “It is important to note that in the pom.xml file you specify the what and not the how. The pom.xml file can also serve as a documentatio...

Java New IO

less than 1 minute read

Notes JDK 1.0 introduced rudimentary I/O facilities for accessing the file system (to create a directory, remove a file, or perform another task), accessi...

IT-Architect

1 minute read

SOA SOA is a set of design principles for building a suite of interoperable, flexible and reusable services based architecture. top-down and bottom-up a...

Algorithm

15 minute read

This page is about key points about Algorithm

Java-Tricky-Tech-Questions.md

19 minute read

What is the difference between Serializable and Externalizable in Java? In earlier version of Java, reflection was very slow, and so serializaing large ob...

Compare-In-Java

6 minute read

Concepts If you implement Comparable interface and override compareTo() method it must be consistent with equals() method i.e. for equal object by equals(...

Java Collections Misc

3 minute read

Difference between equals and deepEquals of Arrays in Java Arrays.equals() method does not compare recursively if an array contains another array on oth...

HashMap in JDK

4 minute read

Hashmap in JDK Some note worth points about hashmap Lookup process Step# 1: Quickly determine the bucket number in which this element may resid...

Java 8 Tips

66 minute read

This blog is listing key new features introduced in Java 8

Back to top ↑

2016

Java GC notes

28 minute read

verbose:gc verbose:gc prints right after each gc collection and prints details about each generation memory details. Here is blog on how to read verbose gc

Hash Code Misc

12 minute read

contract of hashCode : Whenever it is invoked on the same object more than once during an execution of a Java application, the hashCode method must consis...

Angulary Misc

2 minute read

Dependency Injection Angular doesn’t automatically know how you want to create instances of your services or the injector to create your service. You must co...

Random number in java

less than 1 minute read

ThreadLocalRandom, SecureRandm, java.util.Random, java.math.Random

Java new features

less than 1 minute read

JDK Versions JDK 1.5 in 2005 JDK 1.6 in 2006 JDK 1.7 in 2011 JDK 1.8 in 2014 Sun之前风光无限，但是在2010年1月27号被Oracle收购。 在被Oracle收购后对外承诺要回到每2年一个realse的节奏。但是20...

Simpler chronicle of CI(Continuous Integration) “乱弹系列”之持续集成工具

less than 1 minute read

引言 有句话说有人的地方就有江湖，同样，有江湖的地方就有恩怨。在软件行业历史长河（虽然相对于其他行业来说，软件行业的历史实在太短了，但是确是充满了智慧的碰撞也是十分的精彩）中有一些恩怨情愁，分分合合的小故事，比如类似的有，从一套代码发展出来后面由于合同到期就分道扬镳，然后各自发展成独门产品的Sybase DB和微...

浅谈软件单元测试中的“断言” (assert)，从石器时代进步到黄金时代。

less than 1 minute read

大家都知道，在软件测试特别是在单元测试时,必用的一个功能就是“断言”（Assert)，可能有些人觉得不就一个Assert语句，没啥花头，也有很多人用起来也是懵懵懂懂，认为只要是Assert开头的方法，拿过来就用。一个偶然的机会跟人聊到此功能，觉得还是有必要在此整理一下如何使用以及对“断言”的理解。希望可以帮助大家...

Head First Blockchina 1

less than 1 minute read

深入浅出区块链系统：第一章. what you should know about blockchain

Kubernetes 与 Docker Swarm的对比

less than 1 minute read

Kubernetes 和Docker Swarm 可能是使用最广泛的工具，用于在集群环境中部署容器。但是这两个工具还是有很大的差别。

http methods

11 minute read

RFC origion http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1.2)

Spark-vs-Storm

less than 1 minute read

The stark difference among Spark and Storm. Although both are claimed to process the streaming data in real time. But Spark processes it as micro-batches; wh...

微服务

less than 1 minute read

可以想像一下，之前的传统应用系统，像是一个大办公室里面，有各个部门，销售部，采购部，财务部。办一件事情效率比较高。但是也有一些弊端，首先，各部门都在一个房间里。

kibana, view layer of elasticsearch

less than 1 minute read

What’s Kibana kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on...

kibana, view layer of elasticsearch

less than 1 minute read

What’s Kibana kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on...

iConnect

less than 1 minute read

UI HTML5, AngularJS, BootStrap, REST API, JSON Backend Hadoop core (HDFS), Hive, HBase, MapReduce, Oozie, Pig, Solr

Business Analysis

less than 1 minute read

Purpose of BA 带来一些商业价值（收益） 解决业务痛点

Data Structure

2 minute read

Binary Tree A binary tree is a tree in which no node can have more than two children. A property of a binary tree that is sometimes important is that th...

equity trading

less than 1 minute read

Difference between mutal funds and hedge funds

SQL

less than 1 minute read

Differences between not in, not exists , and left join with null

HTTPS/2

less than 1 minute read

concepts

Github page commands notes

less than 1 minute read

404 error for customized domain (such as godday) 404 There is not a GitHub Pages site here. Go to github master branch for gitpages site, manually add CN...

RenMinBi International

1 minute read

RQFII RQFII stands for Renminbi Qualified Foreign Institutional Investor. RQFII was introduced in 2011 to allow qualified foreign institutional investors to ...

Load Balancing

less than 1 minute read

Concepts LVS means Linux Virtual Server, which is one Linux built-in component.

Python

less than 1 minute read

(‘—–Unexpected error:’, <type ‘exceptions.TypeError’>) datetime.datetime.now()

Setup Git in Mint Linux

less than 1 minute read

How to setup Git in Mint Linux =================================================

Microservices vs. SOA

1 minute read

Microservice Services are organized around capabilities, e.g., user interface front-end, recommendation, logistics, billing, etc. Services are small in ...

Java Class Loader

33 minute read

Codecache The maximum size of the code cache is set via the -XX:ReservedCodeCacheSize=N flag (where N is the default just mentioned for the particular com...

Back to top ↑