Linux Tips

Remember, some things have to end for better things to begin.

To kill processes with a port number

If you try to kill processes running in port 1888

kill -9 $(lsof -ti:1888)

Get permission denied error when sudo su (or hyphen in sudo command)

bash: /home/YOURNAME/.bashrc: Permission denied That’s because you didn’t add “-“ hyphen in your sudo command.

The difference between “-“ and “no hyphen” is that the latter keeps your existing environment (variables, etc); the former creates a new environment (with the settings of the actual user, not your own).

The hyphen has two effects:

1) switches from the current directory to the home directory of the new user (e.g., to /root in the case of the root user) by logging in as that user

2) changes the environmental variables to those of the new user as dictated by their ~/.bashrc. That is, if the first argument to su is a hyphen, the current directory and environment will be changed to what would be expected if the new user had actually logged on to a new session (rather than just taking over an existing session).

To delete lines in files contain pattern

sed -i '/.*167\=OPT.*/d' testdata.txt

to select only only one element value of XML file :

grep -oPm1 “(?<=)[^<]+"

To check Linux release name

cat /etc/os-release

How to check whether your linux is 32bit or 64 bit

To run “arch” command, this is similar to “uname -m” , it prints to the screen whether your system is running 32-bit (“i686”) or 64-bit (“x86_64”).

convert line ending to unix (sometimes git submit is dos format)

dos2unix the_script_file_name

To check redhat Linux version

cat  /etc/redhat-release

To list all users in linux

cat /etc/passwd

Show IP address in Linux

$ifconfig
eth0      Link encap:Ethernet  HWaddr 00:50:56:9B:19:81
          inet addr:133.14.16.5  Bcast:133.14.16.255  Mask:255.255.255.0

Check system resource

execute `cat /proc/cpuinfo` and `free -m` to gain information about the server’s CPU and memory.

chmod command

From one to four octal digits Any omitted digits are assumed to be leading zeros.

The first digit = selects attributes for the set user ID (4) and set group ID (2) and save text image (1)S The second digit = permissions for the user who owns the file: read (4), write (2), and execute (1) The third digit = permissions for other users in the file’s group: read (4), write (2), and execute (1) The fourth digit = permissions for other users NOT in the file’s group: read (4), write (2), and execute (1)

The octal (0-7) value is calculated by adding up the values for each digit User (rwx) = 4+2+1 = 7 Group(rx) = 4+1 = 5 World (rx) = 4+1 = 5 chmode mode = 0755

Examples

chmod 400 file - Read by owner
chmod 040 file - Read by group
chmod 004 file - Read by world 

chmod 200 file - Write by owner
chmod 020 file - Write by group
chmod 002 file - Write by world

top

  • enter u, then user id to show only user process
  • Z: global color scheme, e.g. red or green
  • B: global bold for all
  • z: show color, then b to hightlight, and x highlight sort fidl, y highlight running tasks
  • #3: show only 3 threads
  • c: show command line
  • F: sort, e.g. Fk sort by CPU%
  • R: reverse order

Sample config files

.vimrc


set number
set incsearch
set hlsearch
syntax on
colorscheme desert

==== screenrc =

https://gist.githubusercontent.com/ChrisWills/1337178/raw/8275b66c3ea86a562cdaa16f1cc6d9931d521e1b/.screenrc-main-example

# GNU Screen - main configuration file 
# All other .screenrc files will source this file to inherit settings.
# Author: Christian Wills - cwills.sys@gmail.com

# Allow bold colors - necessary for some reason
attrcolor b ".I"

# Tell screen how to set colors. AB = background, AF=foreground
termcapinfo xterm 'Co#256:AB=\E[48;5;%dm:AF=\E[38;5;%dm'

# Enables use of shift-PgUp and shift-PgDn
termcapinfo xterm|xterms|xs|rxvt ti@:te@

# Erase background with current bg color
defbce "on"

# Enable 256 color term
term xterm-256color

# Cache 30000 lines for scroll back
defscrollback 30000

# New mail notification
# backtick 101 30 15 $HOME/bin/mailstatus.sh

hardstatus alwayslastline 
# Very nice tabbed colored hardstatus line
hardstatus string '%{= Kd} %{= Kd}%-w%{= Kr}[%{= KW}%n %t%{= Kr}]%{= Kd}%+w %-= %{KG} %H%{KW}|%{KY}%101`%{KW}|%D %M %d %Y%{= Kc} %C%A%{-}'

# change command character from ctrl-a to ctrl-b (emacs users may want this)
#escape ^Bb

# Hide hardstatus: ctrl-a f 
bind f eval "hardstatus ignore"
# Show hardstatus: ctrl-a F
bind F eval "hardstatus alwayslastline"

====================.bashrc ==========

# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
        . /etc/bashrc
fi
# source ./prompt
export PS1=''

export PS1='[\e[104mLight blue \u \A\]$ '

export PS1="\[\e[32m\]\u@\h \d \t \w \[\e[m\] \\$"

\e[104mLight blue

# Welcome message
echo -ne "Good Morning ! It's "; date '+%A, %B %-d %Y'
echo -e "And now your moment of Zen:"; fortune
echo
echo "Hardware Information:"
sensors  # Needs: 'sudo apt-get install lm-sensors'
uptime   # Needs: 'sudo apt-get install lsscsi'
free -m

# User specific aliases and functions

PS1='\[`[ $? = 0 ] && X=2 || X=1; tput setaf $X`\]\h\[`tput sgr0`\]:$PWD\n\$ '

============vimrc =========
set number
set incsearch
set hlsearch

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

grep -v "unwanted_word" file | grep XXXXXXXX

// find command exclude “permission denied”
$ find . -name "java" 2>/dev/null

Passwordless connection in putty

1. Generate Public & private key pair by keygen
2. Log into Linux, nano .ssh/authorized_keys and paste the public key
3. Save the private key in putty and load it in Putty session

find java files older than 3 days

find . -name "*.java" -atime -3d

Remove capitalization

sed -ie 's/Return /return /g' ReverseString.java 

replace string in files

#grep -r "pack.*me" .
sed -ie 's/package.*me.*;/package com.todzhang;/g' *.java
sed -ie 's/package me.todzhang;/package com.todzhang;/g' ~/dev/git/algo/algoWS/src/main/java/com/todzhang/*.java

to update layout from post to posts for jekyll in batch

sed -ie ‘s/layout: posts/layout: posts/g’ _posts/2016-06-03*.md

layout: posts

create directory hierarchy via path

mkdir -p ~/abc/def/egg

-p means create intermediary folders, if not exist. Those intermediary folders with permission 777

lsof to locate whether/who allocated port 8080

lsof means list open files.

lsof -n -P -i | grep 8080

To get rid of ‘’

Sometimes got “403 Forbidden” error when trying to downalod file via wget, e.g.

$ wget http://www.xmind.net/xmind/downloads/xmind-7.5-update1-macosx.dmg
--2016-09-09 23:27:29--  http://www.xmind.net/xmind/downloads/xmind-7.5-update1-macosx.dmg
Resolving www.xmind.net... 23.23.188.223
Connecting to www.xmind.net|23.23.188.223|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2016-09-09 23:27:29 ERROR 403: Forbidden.

To solve this problem, using following syntax, adding -U xx

wget -U 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4' http://www.xmind.net/xmind/downloads/xmind-7.5-update1-macosx.dmg

case insensitive ls command in bash

Update .bashrc or current active window

shopt -s nocaseglob

one line command to download and extract files

$cd /tmp;curl https://www.kernel.org/pub/linux/utils/util-linux/v2.24/util-linux-2.24.tar.gz	| tar -zxf-;cd	util-linux-2.24;

Redirect request for HTTP 3xxx code

curl -LSso ~/.vim/autoload/pathogen.vim https://tpo.pe/pathogen.vim

-L means redirect upon HTTP code 3xxx -Ss work together to make the curl show errors if there are -o means output to a specified file, rather than stdout

search files contains keyword

grep -ri 'architect' . | awk -F ':' '{print $1}'

Show Linux kernel and name

lsb_release

lsb means Linux Standard Base , -a means print all information

lsb_release -a -u

will output

phray@phray-VirtualBox ~ $ lsb_release -a -u
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 14.04 LTS
Release:	14.04
Codename:	trusty

/etc/os-release

cat /etc/os-release

will output Following

phray@phray-VirtualBox ~ $ cat /etc/os-release
NAME="Ubuntu"
VERSION="14.04.2 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.2 LTS"
VERSION_ID="14.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"

Following is the command found in docker.sh

lsb_dist=$(lsb_release -a -u 2>&1 | tr '[:upper:]' '[:lower:]' | grep -E 'id' | cut -d ':' -f 2 | tr -d '[[:space:]]')

show the 2nd column

lsb_release --Codename | cut -f2

gpsswd

DESCRIPTION gpasswd is used to administer the /etc/group file (and /etc/gshadow file if compiled with SHADOWGRP defined). Every group can have administrators, members and a password. System administrator can use -A option to define group administrator(s) and -M option to define members and has all rights of group administrators and members.

Notes about group passwords Group passwords are an inherent security problem since more than one person is permitted to know the password. However, groups are a useful tool for permitting co-operation between different users.

OPTIONS Group administrator can add and delete users using -a and -d options respectively. Administrators can use -r option to remove group password. When no password is set only group members can use newgrp to join the group. Option -R disables access via a password to the group through newgrp command (however members will still be able to switch to this group).

gpasswd called by a group administrator with group name only prompts for the group password. If password is set the members can still newgrp(1) without a password, non-members must supply the password.

FILES Tag Description /etc/group Group account information. /etc/gshadow Secure group account information.

sudo gpasswd -a USER docker

Compare files difference in two folders

diff -rq ~/dev/pa ~/dev/hexo/myblog/source/_posts

This used option -r (recursive) and -q quite, means only show differences

To execut shell/unix command within vim

:~ls -lt

To open find result with sublime

find . -name "*Linux*.md" | xargs sublime 
find . -name "*Linux*.md" | xargs sublime ~ # open in new Sublime window

To vim/vim edit directly on file output by find command

find . -name "*tmux*" -exec vim {} \;

Be advised you may experience following error message

find: missing argument to `-exec’

actually you should add a slash in front of semi colon

quite mode in apt-get

  • apt-get will in verbose mode
  • apt-get -q will be in less verbose , a.k.a quite mode
  • apt-get -qq in extreme least verbose mode

2022

Minium Workable Mvp Vimrc

”—————————————————————- “ 4. User interface “—————————————————————- “ Set X lines to the cursor when moving vertically set scrolloff=0

Linux Tips

Remember, some things have to end for better things to begin.

Back to Top ↑

2021

How to user fire extinguisher

Summary As you know, staff and your safety is paramount. So what if emergency take place, such as fire in office, how to help yourself and your colleagues by...

Deep dive into Kubernetes Client API

Summary To talk to K8s for getting data, there are few approaches. While K8s’ official Java library is the most widely used one. This blog will look into thi...

Whitelabel Error Page

Summary Whitelabel Error Page is the default error page in Spring Boot web app. It provide a more user-friently error page whenever there are any issues when...

Google maps no photos reviews

Summary I found a weird problem of the app Google Maps of my Oppo Android phone. That’s when you search a place in Google map, say “Central Park”, ideally th...

Debts in a nutshell

A debt security represents a debt owed by the issuer to an investor. Here, the investor acts as a lender to the issuer which may be a government, organisatio...

Back to Top ↑

2020

Debug Stuck IntelliJ

What happened to a debug job hanging in IntelliJ (IDEAS) IDE? You may find when you try to debug a class in Intellij but it stuck there and never proceed, e....

Awesome Kotlin

Difference with Scala Kotlin takes the best of Java and Scala, the response times are similar as working with Java natively, which is a considerable advantag...

JVM热身

此文是作者英文原文的翻译文章,英文原文在:http://todzhang.com/posts/2018-06-10-jvm-warm-up/

Mock in kotlin

Argument Matching & Answers For example, you have mocked DOC with call(arg: Int): Intfunction. You want to return 1 if argument is greater than 5 and -1 ...

Mock in kotlin

Argument Matching & Answers For example, you have mocked DOC with call(arg: Int): Intfunction. You want to return 1 if argument is greater than 5 and -1 ...

Curl

Linux Curl command

AOP

The concept of join points as matched by pointcut expressions is central to AOP, and Spring uses the AspectJ pointcut expression language by default.

Micrometer notes

As a general rule it should be possible to use the name as a pivot. Dimensions allow a particular named metric to be sliced to drill down and reason about th...

Awesome SSL certificates and HTTPS

What’s TLS TLS (Transport Layer Security) and its predecessor, SSL (Secure Sockets Layer), are security protocols designed to secure the communication betwee...

JVM warm up by Escape Analysis

Why JVM need warm up I don’t know how and why you get to this blog. But I know the key words in your mind are “warm” for JVM. As the name “warm up” suggested...

Java Concurrent

This blog is about noteworthy pivot points about Java Concurrent Framework Back to Java old days there were wait()/notify() which is error prone, while fr...

Back to Top ↑

2019

Conversations with God

Feelings is the language of the soul. If you want to know what’s true for you about something, look to how your’re feeling about.

Kafka In Spring

Enable Kafka listener annotated endpoints that are created under the covers by a AbstractListenerContainerFactory. To be used on Configuration classes as fol...

Mifid

FX Spot is not covered by the regulation, as it is not considered to be a financial instrument by ESMA, the European Union (EU) regulator. As FX is considere...

Foreign Exchange

currency pairs Direct ccy: means USD is part of currency pair Cross ccy: means ccy wihtout USD, so except NDF, the deal will be split to legs, both with...

Back to Top ↑

2018

Guice

A new type of Juice Put simply, Guice alleviates the need for factories and the use of new in your Java code. Think of Guice’s @Inject as the new new. You wi...

YAML

Key points All YAML files (regardless of their association with Ansible or not) can optionally begin with — and end with …. This is part of the YAML format a...

Sudo in a Nutshell

Sudo in a Nutshell Sudo (su “do”) allows a system administrator to give certain users (or groups of users) the ability to run some (or all) commands as root...

Zoo-keeper

ZK Motto the motto “ZooKeeper: Because Coordinating Distributed Systems is a Zoo.”

Cucumber

Acceptance testing vs unit test It’s sometimes said that unit tests ensure you build the thing right, whereas acceptance tests ensure you build the right thi...

akka framework of scala

philosophy The actor model adopts the philosophy that everything is an actor. This is similar to the everything is an object philosophy used by some object-o...

Apache Camel

Camel’s message model In Camel, there are two abstractions for modeling messages, both of which we’ll cover in this section. org.apache.camel.Message—The ...

JXM

Exporting your beans to JMX The core class in Spring’s JMX framework is the MBeanExporter. This class is responsible for taking your Spring beans and registe...

Solace MQ

Solace PubSub+ It is a message broker that lets you establish event-driven interactions between applications and microservices across hybrid cloud environmen...

Apigee

App deployment, configuration management and orchestration - all from one system. Ansible is powerful IT automation that you can learn quickly.

Ansible

Ansible: What Is It Good For? Ansible is often described as a configuration management tool, and is typically mentioned in the same breath as Chef, Puppet, a...

flexbox

How Flexbox works — explained with big, colorful, animated gifs

KDB

KDB However kdb+ evaluates expressions right-to-left. There are no precedence rules. The reason commonly given for this behaviour is that it is a much simple...

Agile and SCRUM

Key concept In Scrum, a team is cross functional, meaning everyone is needed to take a feature from idea to implementation.

Strategy-Of-Openshift-Releases

Release & Testing Strategy There are various methods for safely releasing changes to Production. Each team must select what is appropriate for their own ...

NodeJs Notes

commands to read files var lineReader = require(‘readline’).createInterface({ input: require(‘fs’).createReadStream(‘C:\dev\node\input\git_reset_files.tx...

CORS :Cross-Origin Resource Sharing

Cross-Origin Request Sharing - CORS (A.K.A. Cross-Domain AJAX request) is an issue that most web developers might encounter, according to Same-Origin-Policy,...

ngrx

Why @Effects? In a simple ngrx/store project without ngrx/effects there is really no good place to put your async calls. Suppose a user clicks on a button or...

iOS programming

View A view is also a responder (UIView is a subclass of UIResponder). This means that a view is subject to user interactions, such as taps and swipes. Thus,...

Back to Top ↑

2017

cloud computering

openshift vs openstack The shoft and direct answer is `OpenShift Origin can run on top of OpenStack. They are complementary projects that work well together....

cloud computering

Concepts Cloud computing is the on-demand demand delivery of compute database storage applications and other IT resources through a cloud services platform v...

Redux

whats @Effects You can almost think of your Effects as special kinds of reducer functions that are meant to be a place for you to put your async calls in suc...

reactive programing

The second advantage to a lazy subscription is that the observable doesn’t hold onto data by default. In the previous example, each event generated by the in...

Container

The Docker project was responsible for popularizing container development in Linux systems. The original project defined a command and service (both named do...

promise vs observiable

The drawback of using Promises is that they’re unable to handle data sources that produce more than one value, like mouse movements or sequences of bytes in ...

JDK source

interface RandomAccess Marker interface used by List implementations to indicate that they support fast (generally constant time) random access. The primary ...

SSH SFTP

Secure FTP SFTP over FTP is the equivalant of HTTPS over HTTP, the security version

AWS Tips

After establishing a SSH session, you can install a default web server by executing sudo yum install httpd -y. To start the web server, type sudo service htt...

Oracle

ORA-12899: Value Too Large for Column

Kindle notes

#《亿级流量网站架构核心技术》目录一览 TCP四层负载均衡 使用Hystrix实现隔离 基于Servlet3实现请求隔离 限流算法 令牌桶算法 漏桶算法 分布式限流 redis+lua实现 Nginx+Lua实现 使用sharding-jdbc分库分表 Disruptor+Redis...

Java Security Notes

Java Security well-behaved: programs should be prevent from consuming too much system resources

R Language

s<-read.csv("C:/Users/xxx/dev/R/IRS/SHH_SCHISHG.csv") # aggregate s2<-table(s$Original.CP) s3<-as.data.frame(s2) # extract by Frequency ordered s3...

SSH and Cryptography

SFTP versus FTPS SS: Secure Shell An increasing number of our customers are looking to move away from standard FTP for transferring data, so we are ofte...

Eclipse notes

How do I remove a plug-in? Run Help > About Eclipse > Installation Details, select the software you no longer want and click Uninstall. (On Macintosh i...

Maven-Notes

Maven philosophy “It is important to note that in the pom.xml file you specify the what and not the how. The pom.xml file can also serve as a documentatio...

Java New IO

Notes JDK 1.0 introduced rudimentary I/O facilities for accessing the file system (to create a directory, remove a file, or perform another task), accessi...

IT-Architect

SOA SOA is a set of design principles for building a suite of interoperable, flexible and reusable services based architecture. top-down and bottom-up a...

Algorithm

This page is about key points about Algorithm

Java-Tricky-Tech-Questions.md

What is the difference between Serializable and Externalizable in Java? In earlier version of Java, reflection was very slow, and so serializaing large ob...

Compare-In-Java

Concepts If you implement Comparable interface and override compareTo() method it must be consistent with equals() method i.e. for equal object by equals(...

Java Collections Misc

Difference between equals and deepEquals of Arrays in Java Arrays.equals() method does not compare recursively if an array contains another array on oth...

HashMap in JDK

Hashmap in JDK Some note worth points about hashmap Lookup process Step# 1: Quickly determine the bucket number in which this element may resid...

Java 8 Tips

This blog is listing key new features introduced in Java 8

Back to Top ↑

2016

Java GC notes

verbose:gc verbose:gc prints right after each gc collection and prints details about each generation memory details. Here is blog on how to read verbose gc

Hash Code Misc

contract of hashCode : Whenever it is invoked on the same object more than once during an execution of a Java application, the hashCode method must consis...

Angulary Misc

Dependency Injection Angular doesn’t automatically know how you want to create instances of your services or the injector to create your service. You must co...

Java new features

JDK Versions JDK 1.5 in 2005 JDK 1.6 in 2006 JDK 1.7 in 2011 JDK 1.8 in 2014 Sun之前风光无限,但是在2010年1月27号被Oracle收购。 在被Oracle收购后对外承诺要回到每2年一个realse的节奏。但是20...

Simpler chronicle of CI(Continuous Integration) “乱弹系列”之持续集成工具

引言 有句话说有人的地方就有江湖,同样,有江湖的地方就有恩怨。在软件行业历史长河(虽然相对于其他行业来说,软件行业的历史实在太短了,但是确是充满了智慧的碰撞也是十分的精彩)中有一些恩怨情愁,分分合合的小故事,比如类似的有,从一套代码发展出来后面由于合同到期就分道扬镳,然后各自发展成独门产品的Sybase DB和微...

浅谈软件单元测试中的“断言” (assert),从石器时代进步到黄金时代。

大家都知道,在软件测试特别是在单元测试时,必用的一个功能就是“断言”(Assert),可能有些人觉得不就一个Assert语句,没啥花头,也有很多人用起来也是懵懵懂懂,认为只要是Assert开头的方法,拿过来就用。一个偶然的机会跟人聊到此功能,觉得还是有必要在此整理一下如何使用以及对“断言”的理解。希望可以帮助大家...

Kubernetes 与 Docker Swarm的对比

Kubernetes 和Docker Swarm 可能是使用最广泛的工具,用于在集群环境中部署容器。但是这两个工具还是有很大的差别。

http methods

RFC origion http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1.2)

Spark-vs-Storm

The stark difference among Spark and Storm. Although both are claimed to process the streaming data in real time. But Spark processes it as micro-batches; wh...

微服务

可以想像一下,之前的传统应用系统,像是一个大办公室里面,有各个部门,销售部,采购部,财务部。办一件事情效率比较高。但是也有一些弊端,首先,各部门都在一个房间里。

kibana, view layer of elasticsearch

What’s Kibana kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on...

kibana, view layer of elasticsearch

What’s Kibana kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on...

iConnect

UI HTML5, AngularJS, BootStrap, REST API, JSON Backend Hadoop core (HDFS), Hive, HBase, MapReduce, Oozie, Pig, Solr

Data Structure

Binary Tree A binary tree is a tree in which no node can have more than two children. A property of a binary tree that is sometimes important is that th...

Something about authentication

It’s annoying to keep on repeating typing same login and password when you access multiple systems within office or for systems in external Internet. There a...

SQL

Differences between not in, not exists , and left join with null

Github page commands notes

404 error for customized domain (such as godday) 404 There is not a GitHub Pages site here. Go to github master branch for gitpages site, manually add CN...

RenMinBi International

RQFII RQFII stands for Renminbi Qualified Foreign Institutional Investor. RQFII was introduced in 2011 to allow qualified foreign institutional investors to ...

Load Balancing

Concepts LVS means Linux Virtual Server, which is one Linux built-in component.

Python

(‘—–Unexpected error:’, <type ‘exceptions.TypeError’>) datetime.datetime.now()

Microservices vs. SOA

Microservice Services are organized around capabilities, e.g., user interface front-end, recommendation, logistics, billing, etc. Services are small in ...

Java Class Loader

Codecache The maximum size of the code cache is set via the -XX:ReservedCodeCacheSize=N flag (where N is the default just mentioned for the particular com...

Back to Top ↑