用10几行代码自己写个人脸识别程序
用10几行代码自己写个人脸识别程序
CV (Computer Vision)
最近在研究CV的一些开源库(OpenCV)，有一个体会就是在此领域，除了一些非常学术的_机器学习_, 深度学习_等概念外，其实还有一些很有趣的_现实的_应用场景。比如之前很流行的微软的 https://how-old.net, 你使用自己指定或者上传的照片进行面部识别_猜年龄。 如下图所示：
细想一下这个很吸引眼球的程序，其实技术本身打散了就包括两大块，一是从图片中扫描并进行面部识别，二是对找到的人脸根据算法去猜个年龄。大家可以猜猜实现第一个功能需要多少核心代码量？其实不用上万行，在这里我就使用短短几行代码（去除空格换行什么的，有效代码只要10行）就实现一个_高大上_面部识别的功能。在此文容我细述一下具体实现代码以及我对机器识别图像领域技术的理解。
面部识别,刷脸
_人脸识别_技术大家应该都不陌生，之前大家使用的数码相机，或者现在很多手机自带的相机都有人脸识别的功能。其效果就像是下图这样。近的看，_剁手节_刚刚过了没有多久 , 背后的马老板一直在力推的刷脸支付也是一个此领域的所谓“黑科技”。比如在德国汉诺威电子展上，马云用支付宝“刷脸”买了一套纪念邮票。人脸识别应用市场也从爆发。随后，各大互联网巨头也纷纷推出了刷脸相关的应用。
如果要加个定义，人脸识别又叫做人像识别、面部识别，是一种通过用摄像机或摄像头采集含有人脸的图像或视频流，并自动在图像中检测和跟踪人脸，进而对检测到的人脸进行脸部的一系列相关技术。
我的十行代码程序
OK，长话短说，先上 干货 ，下面就是此程序的_带注释_ 版本，完整的程序以及相关配套文件可以在 这个github库 https://github.com/CloudsDocker/pyFacialRecognition 中找到，有兴趣可以_fork_ 下来玩玩。下面是整个程序的代码样子，后面我会逐行去解释分析。
就这短短的十行代码代码？seriously？“有图有真相”，我们先来看下运行的效果：
首先是原始的图片
运行程序后识别出面部并高亮显示的结果
请注意 K歌二人组 的脸上的红色框框，这就是上面十行代码的成果。
代码解析
准备工作
因为此程序使用是的Python,因此你需要去安装Python。这里就不赘述了。除此之外，还需要安装 OpenCV (http://opencv.org/downloads.html)。 多说一句,这个 OpenCV正如其名，是一个开源的机器识别的深度学习框架。这是Intel（英特尔）实验室里的一个俄罗斯团队创造的，目前在开源社区非常的活跃。
特别提一下，对于Mac的用户，推荐使用brew去安装 （下面第一条语句可能会执行报错，我当时也是搞了好久。如果遇到第一条命令不过可以通过文尾的方式联系作者）
brew tap homebrew/science
brew install opencv
安装完成之后,在python的命令行中输入如下代码验证，如果没有报错就说明安装好了。
>>> import cv2
程序代码“庖丁解牛”
# -*- coding: utf-8 -*-
import cv2,sys
- 由于这里注释及窗口标题中使用了中文，因此加上utf-8字符集的支持
- 引入Opencv库以及Python的sys内建库，用于解析输入的图片参数
inputImageFile=sys.argv[1]
- 在运行程序时将需要测试的照片文件名作为一个参数传进来
faceClassifier=cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
- 加载OpenCV中自带预先培训好的人脸识别层级分类器 HAAR Casscade Classifier，这个会用来对我们输入的图片进行人脸判断。
这里有几个在深度学习及机器图像识别领域中的几个概念，稍微分析一下，至于深入的知识，大家可以自行搜索或者联系作者。
Classifer
在机器深度学习领域，针对识别不同物体都有不同的classifier,比如有的classifier来识别洗车，还有识别飞机的classifier，有classifier来识别照片中的笑容，眼睛等等。而我们这个例子是需要去做人脸识别，因此需要一个面部识别的classifier。
物体识别的原理
一般来说，比如想要机器学习着去识别“人脸”，就会使用大量的样本图片来事先培训，这些图片分为两大类，positive和negative的，也就是分为包“含有人脸”的图片和“不包含人脸”的图片，这样当使用程序去一张一张的分析这些图片，然后分析判断并对这些图片“分类” (classify),即合格的图片与不合格的图片，这也就其为什么叫做 classifier ， 这样学习过程中积累的”知识”，比如一些判断时的到底临界值多少才能判断是positive还是negative什么的，都会存储在一个个XML文件中，这样使用这些前人经验（这里我们使用了 哈尔 分类器）来对新的图片进行‘专家判断’分析，是否是人脸或者不是人脸。
Cascade
这里的 Cascade是 层级分类器 的意思。为什么要 分层 呢？刚才提到在进行机器分析照片时，其实是对整个图片从上到下，从左到右，一个像素一个像素的分析，这些分析又会涉及很多的 特征分析 ，比如对于人脸分析就包含识别眼睛，嘴巴等等，一般为了提高分析的准确度都需要有成千上万个特征，这样对于每个像素要进行成千上万的分析，对于整个图片都是百万甚至千万像素，这样总体的计算量会是个天文数字。但是，科学家很聪明，就想到分级的理念，即把这些特征分层，这样分层次去验证图片，如果前面层次的特征没有通过，对于这个图片就不用判断后面的特征了。这有点像是系统架构中的 FF (Fail Fast),这样就提高了处理的速度与效率。
objImage=cv2.imread(inputImageFile)
- 使用OpenCV库来加载我们传入的测试图片
cvtImage=cv2.cvtColor(objImage,cv2.COLOR_BGR2GRAY)
- 首先将图片进行灰度化处理，以便于进行图片分析。这种方法在图像识别领域非常常见，比如在进行验证码的机器识别时就会先灰度化，去除不相关的背景噪音图像，然后再分析每个像素，以便抽取出真实的数据。不对针对此，你就看到非常多的验证码后面特意添加了很多的噪音点，线，就是为了防止这种程序来灰度化图片进行分析破解。
foundFaces=faceClassifier.detectMultiScale(cvtImage,scaleFactor=1.3,minNeighbors=9,minSize=(50,50),flags = cv2.cv.CV_HAAR_SCALE_IMAGE)
- 执行detectMultiScale方法来识别物体，因为我们这里使用的是人脸的cascade classifier分类器，因此调用这个方法会来进行面部识别。后面这几个参数来设置进行识别时的配置，比如
- scaleFactor: 因为在拍照，尤其现在很多都是自拍，这样照片中有的人脸大一些因为离镜头近，而有些离镜头远就会小一些，因为这个参数用于设置这个因素，如果你在使用不同的照片时如果人脸远近不同，就可以修改此参数，请注意此参数必须要大于1.0
- minNeighbors: 因为在识别物体时是使用一个移动的小窗口来逐步判断的，这个参数就是决定是不是确定找到物体之前需要判断多少个周边的物体
- minSize：刚才提到识别物体时是合作小窗口来逐步判断的，这个参数就是设置这个小窗口的大小
print(" 在图片中找到了 {} 个人脸".format(len(foundFaces)))
- 显示出查找到多少张人脸，需要提到的识别物体的方法返回的一个找到的物体的位置信息的列表，因此使用 len 来打印出找到了多少物体。
for (x,y,w,h) in foundFaces:
cv2.rectangle(objImage,(x,y),(x+w,y+h),(0,0,255),2)
- 遍历发现的“人脸”，需要说明的返回的是由4部分组成的位置数据，即这个“人脸”的横轴，纵轴坐标，宽度与高度。
- 然后使用 OpenCV 提供的方法在原始图片上画出个矩形。其中 (0,0,255) 是使用的颜色，这里使用的是R/G/B的颜色表示方法，比如 (0,0,0)表示黑色，(255,255,255)表示白色，有些网页编程经验的程序员应该不陌生。
cv2.imshow(u'面部识别的结果已经高度框出来了。按任意键退出'.encode('gb2312'), objImage)
cv2.waitKey(0)
- 接下来是使用 opencv 提供的imshow方法来显示这个图片，其中包括我们刚刚画的红色的识别的结果
- 最后一个语句是让用户按下键盘任意一个键来退出此图片显示窗口
总结
好了，上面是这个程序的详细解释以及相关的知识的讲解。其实这个只是个_抛砖引玉_的作用，还用非常多的应用场景，比如程序解析网页上的图片验证码，雅虎前几个月开源的 NSFW, Not Suitable for Work (NSFW)，即判断那些不适合工作场所的图片，内容你懂的。 :-)
最后，再提一下，所有这些源代码及相关文件都开源在 https://github.com/CloudsDocker/pyFacialRecognition ，在fork并下载到本地后执行下面代码来测试运行
git clone https://github.com/CloudsDocker/pyFacialRecognition.git
cd pyFacialRecognition
./run.sh
如果有任何建议或者想法，请联系我。
联系我：
- phray.zhang@gmail.com (email/邮件，whatsapp, linkedin)
- helloworld_2000 (wechat/微信)
- 微博: cloudsdocker
- github
- [简书 jianshu]（http://www.jianshu.com/users/a9e7b971aafc）
- 微信公众号：vibex
Reference
2024
Awesome-awk-tools Simplicity is the ultimate sophistication
“Simplicity is the ultimate sophistication.” - Leonardo da Vinci
When Your Retry Mechanism Doesn’t Retry - A Tale of String Matching Gone Wrong
The biggest room in the world is the room for improvement. — Helmut Schmidt
Avoiding Data Loss - S3 Lifecycle Rules During Terraform Version Migrations
Your past is a lesson. Not a life sentence. Forgive yourself and focus on the future. -Mel Robbins
The Great Migration – Moving Azure Web App and App Service Plan Across Subscriptions
A younger brother knows his older brother better than anyone else.
awesome mr W
You are not a drop in the ocean, you are the entire ocean in a drop.
Mastering Date Formatting in Bash: A Developer’s Guide
A younger brother knows his older brother better than anyone else.
Kubenetes Zero to Hero
You are not a drop in the ocean, you are the entire ocean in a drop.
Unraveling The Mystery Nested Sql Comments In Vscode
Unraveling the Mystery of Nested SQL Comments in VS Code Have you ever found yourself staring at a sea of incorrectly highlighted SQL code in Visual Studio C...
Flyway Self Healing
how to let your flyway database scheme migrate more robustly and self healing
Flyway Self Healing
how to let your flyway database scheme migrate more robustly and self healing
Lock Wait Timeout Exceptions and Data Persistence Issues in Spring Boot and Hibernate
If you can make your hobby your profession, you never have to work another day in your life. —Anonymous
Unlocking SQL Superpowers-> How CTEs Will Transform Your Database Queries
“Stress is like a pulse, if you have it you are alive.” — Steve Maraboli
Why Hibernate Still Logs SQL Even When Disabled in application.yaml
Good leadership consists of doing less and being more. —Dave Ramsey
The Curious Case of Azure Key Vault Defender Alerts - When Security Settings Play Hide and Seek
A leader takes people where they want to go. A great leader takes people where they don’t necessarily want to go, but ought to be. —Rosalynn Carter, forme...
Streamline Your Workflow by Fastest way to run Maven Builds with a Keyboard Shortcut in IntelliJ
A leader takes people where they want to go. A great leader takes people where they don’t necessarily want to go, but ought to be. —Rosalynn Carter, forme...
使用 c3p0 连接池解决 Spring Boot 中断的数据库连接问题（解决 Spring Boot 中断的数据库连接问题）
一旦你知道答案，一切都会变得简单。” —— 戴夫·梅吉（Dave Magee）
Resolving Disconnected Database Connections in Spring Boot with c3p0 Connection Pool
“Everything is easy, once you know the answer. —Dave Magee
IntelliJ sudden crashed of compile error MapStruct or Kotlin
Life begins at the edge of the comfort zone
Deep dive for word press preview nonce
One must learn by doing the thing; for though you think you know it, you have no certainty, until you try. —Sophocles
你不了解的word press 的 preview nonce
One must learn by doing the thing; for though you think you know it, you have no certainty, until you try. —Sophocles
Notes and pitfalls for redis development
A younger brother knows his older brother better than anyone else.
2023
Awesome Jq For Coders
Mastering JSON Data Manipulation with jq: A Comprehensive Guide
Awesome Xlookup Over Vlookup
XLOOKUP vs. VLOOKUP: Excel’s Dynamic Duo for Data Lookup
Ports Discovery On Hosts
To find out the port numbers running in servers
Troubleshoot Mariadb In Linux
The simplest way to check an mariadb is runnning systemctl status mariadb
Az Cli
To run commands in VMs in Azure
From filter to CNN (Convolutional Network)
The biggest room in the world is the room for improvement. Filters in Convolutional Neural Networks (CNNs) In the context of convolutional neural net...
Unlocking Network Secrets A MacBook Traceroute Tutorial
A younger brother knows his older brother better than anyone else.
How to Test Logging Output in JUnit
A younger brother knows his older brother better than anyone else.
Cheap and flexible computing
whether it seems possible or not - go for it Cheaper X 2 to EC2, to use Fargate Spot With Fargate Spot you can run interruption tolerant Amazon ECS t...
How Guru to use Capturing Groups in Python Regular Expressions
A dream deferred is a dream denied. -Langston Hughes
Composition and Aggregation in Object-Oriented Modeling
“The past does not equal the future unless you live there.” - Tony Robbins
Exploring the useRequest Hook from ahooks
“The best way to predict the future is to invent it.” - Alan Kay
Understanding Python’s Late Binding Behavior A Deep Dive
“Hang Out with People Who are Better than You.” — Warren Buffett
Understanding React export a Component
A young idler, an old beggar. - William Shakespeare Understanding React export a Component In this blog post, we will dive into the code of the RepoU...
why use mid = (low + high) // 2 but not (high-low)//2
“Don’t let yesterday take up too much of today.” - Will Rogers
Introduction to Generator Expressions in Python
“It always seems impossible until it’s done.” - Nelson Mandela
Introduction to Generator Expressions in Python
We never lose friends but just start to find real ones. - William Shakespeare
The Curious Case of ‘localhost’ vs ‘127.0.0.1’ in MySQL Connections
Everybody may not to be famous but everybody can be great. “The Curious Case of ‘localhost’ vs ‘127.0.0.1’ in MySQL Connections” Have you ever encoun...
Understanding Backpropagation in Neural Networks
Your past is a lesson. Not a life sentence. Forgive yourself and focus on the future. -Mel Robbins
Understanding Backpropagation in Neural Networks
Your past is a lesson. Not a life sentence. Forgive yourself and focus on the future. -Mel Robbins
Understanding Confusion Matrix in WEKA
Your past is a lesson. Not a life sentence. Forgive yourself and focus on the future. -Mel Robbins
Useful Shortcut Tips for MacBook Office Workers
A young idler, an old beggar. - William Shakespeare
Understanding d-Separation in Graphical Models
A young idler, an old beggar. - William Shakespeare
How To Split Sql Insert Statement
“Don’t let yesterday take up too much of today.” - Will Rogers
docker-commands-bible
“Don’t let yesterday take up too much of today.” - Will Rogers
Adversarial Search: Unleashing The Power Of AI In Competitive Games
“What you seek is seeking you.” — Rumi
Weird Problem Changed Configurations In Pom Xml Not Work
“I can’t relate to lazy people. We don’t speak the same language.” — Kobe Bryant
AI Basics, Talk About Searches
“What you seek is seeking you.” — Rumi
Xpath Playground Best Practices
A young idler, an old beggar. - William Shakespeare
UUID deep dive
A young idler, an old beggar. - William Shakespeare
Compile Error Java Kotlin Coexist Project In Intellij
The biggest room in the world is the room for improvement. — Helmut Schmidt
Who Is Running On Port 8080
A young idler, an old beggar. - William Shakespeare
Let AI To Manage Stripe Payment
A young idler, an old beggar. - William Shakespeare
RestTemplate Powered Http2
Why HTTP/2 is Better
Compile Error Java Kotlin Coexist Project In Intellij
How to Fine Tune RestTemplate
how-to-travel-in-cairns
大堡礁的一些知识
Compile Error Java Kotlin Coexist Project In Intellij
The root cause is your customized HttpMessageConverter stopped processing of WebSecurity
io_mockk_MockKException__no_answer_found_for
A young idler, an old beggar. - William Shakespeare
which-port-my-service-is-running
Summary As a Java developer, it’s important to know how to find out which port number your Spring service is running on. This information is useful when you ...
How To Install Sonarqube Via Docker
“Hang Out with People Who are Better than You.” — Warren Buffett
how-to-auto-login-for-citrix-receiver-vpn-client
“Hang Out with People Who are Better than You.” — Warren Buffett
pip-install-behind-proxy
Failure of timeout or connection when running pip install
Elk Search Tips
message:/'Invoking SP with quoteContext*werqewr-1234asdf-sdf23-9d83-asdf23*'/
what is StrictHostKeyChecking in ssh
What’s and how to avoid error of the authenticity of host ‘xxx’ can’t be established You can suppress the “The authenticity of host ‘’ can’t be established” ...
know in and out of free command
You are not a drop in the ocean, you are the entire ocean in a drop.
Chinese Verb
知其雄，守其雌 什么意思
Deep dive for errors during Spring Boot Tests
Transaction silently rolled back because it has been marked as rollback-only
Is Import Star Devil
Why using wildcard import is devil
How To Run Testing Multiple Threading
A sample to test concurrent JPA modifications
Stress Test Concurrency Jpa Entity Random Update
A runnable example in Java to create a cucumber test code files to simulate multiple read and write entity via JPA repository
How To Use Aop Test Utils.jpg
What’s purpose of AopTestUtils.getTargetObject()?
A Brief Introduction To Lookbehind And Lookahead In Regular Expressions
“The only way to do great work is to love what you do.” - Steve Jobs
master-spring-properties-injection
“The only way to do great work is to love what you do.” - Steve Jobs
Transaction Commit In Hibernate Jpa
Give me sample to test concurrent JPA modifications
Spring Boot Test In A Nutshell
what’s spring boot test annotation
How To Detach In Jpa
A real sample of using JPA detach
Feature Flag Spring Boot
summary Feature flag library in spring boot
What’s Difference Of Cny And Cnh
what’s difference of CNY and CNH CNY and CNH are both currencies used in China, but they are different in a few important ways:
Hibernate Transaction Management
Details of how hibernate transaction management works
Spring Cloud Masterpiece 10
In spring cloud what’s when to use feign client and when to sue resttemplate
Spring Cloud Master Piece 9
What’s spring cloud config Spring Cloud Config is a distributed configuration server that provides a centralized location to manage external properties for a...
Spring API Gateway Best Practices
Spring API Gateway Best Practices
Splitting A Monolithic Application Into Microservices
Splitting a monolithic application into microservices can be a complex process that requires careful planning and implementation. Here is a high-level approa...
Spring Cloud Master Piece 6
Sample me build a micro service payment system with spring cloud Here’s an example of building a microservice payment system using Spring Cloud:
Difference Between Using Ribbon And A Load Balancer
The main difference between using Ribbon and a Load Balancer is the location of the load balancing logic.
Spring Cloud Master Piece 5
How to add security among micro service in spring boot
Spring Cloud Master Piece 4
How to use service discovery in spring book
Spring Cloud Master Piece 3
Sample me how to build a eureka service discovery
Spring Cloud Master Piece 2
what’s usage of bootstrap yml In a Spring Boot application, the bootstrap.yml (or bootstrap.properties) file is used for configuring the application’s enviro...
Spring Cloud Master Piece 1
what’s API gateway An API Gateway is a key component in microservices architecture that acts as a single entry point for client requests to a microservices-b...
annoying-debug-logs-in-springboot-test
Stop annoying debug logs in spring boot test
how-to-stop-quartz-scheduling-during-springboot-test
how-to-stop-quartz-scheduling-during-springboot-test
Date Is The Most Ignored Treasure In Macbook
“The only way to do great work is to love what you do.” - Steve Jobs
Mysql Operator To Extract Json
“Believe you can and you’re halfway there.” - Theodore Roosevelt
Master Microfrontends
“The only way to do great work is to love what you do.” - Steve Jobs
How To Convert One Monolith Java System To Microservices
Whatever is worth doing is worth doing well.
How To Config JFR Java Flight Control
“Climb the mountains and get their good tidings. Nature’s peace will flow into you as sunshine flows into trees. The winds will blow their own freshness i...
How To Read Jdk Mission Control Report
Live the life you’ve imagined.
Jdk Mission Control Can Not Start In Macbook M1
“Climb the mountains and get their good tidings. Nature’s peace will flow into you as sunshine flows into trees. The winds will blow their own freshness i...
How To Keep Multiple Copy Paste Value In Macbook
“Winning is nice if you don’t lose your integrity in the process.” — Arnold Horshak
Google マップ内の写真のコメントが表示されない
紹介 私は、私のOppo Androidスマートフォンのアプリ「Googleマップ」で奇妙な問題が発生していることに気づきました。Googleマップで特定の場所（例えば「中央公園」）を検索すると、通常、このアプリは公園の写真やコメントリストを表示するはずです。例えば、誰かが公園の芝生や川の写真を投稿し、便利な場所...
Les commentaires des photos ne s’affichent pas dans Google Maps.
Introduction J’ai remarqué un problème étrange avec l’application “Google Maps” de mon téléphone Android Oppo. Lorsque vous recherchez un lieu sur Google Map...
Is Kerberos One Ssl/tls?
Nothing is as easy as it looks.
How To Save Expect Script Run Output To File Locally
Nothing is as easy as it looks.
Refind Java Concurrency
You are not a drop in the ocean, you are the entire ocean in a drop.
Refind Java Solid Principles
You are not a drop in the ocean, you are the entire ocean in a drop.
How To Extract Table Name From Sql By Python
You are not a drop in the ocean, you are the entire ocean in a drop.
How to find non-empty json value in mysql
You are not a drop in the ocean, you are the entire ocean in a drop.
master-cglib-in-java
You are not a drop in the ocean, you are the entire ocean in a drop.
What is shape function in python pandas
An honest days’ work makes for a good night’s sleep.
What is shape function in python pandas
Imagination is the key ingredient to a happy life.
What is default logic in python try except else
Keep an eye on the fruits of your labor.
Not just use git but know how git symbolic-ref work
Superheros come in all shapes and sizes.
Fix rejection error in Hexo
The heart can see what is invisible to the eye.
Guide to code productively, get more time back for you
The heart can see what is invisible to the eye.
Is Fibonacci sequence that starts with 0 or 1
The best way to predict the future is to create it.
To increase your productivity 10 times, learn expect and read this blog
Som are born beautiful. The rest of us have to work at it.
Treasure Bowl for SQL, helpful for your daily database jobs
Don’t be greedy. Half of something is better than all nothing.
How to fix most permission issues when using Git
The best way to predict the future is to create it.
One killer page to fix most permission issues when using Git
The best way to predict the future is to create it.
How to check your CPU model and Linux distribution in your AWS VM
Lift is short, enjoy the ride.
2022
Magic-in-Micronaut-JPA
The best way to predict the future is to create it.
谷歌地图里面照片的评论和照片在华为手机里面显示不出来
枝上柳棉吹又少, 天涯何处无芳草. –苏轼
Cannot find symbol class Generated or var
The best way to predict the future is to create it.
GraphQL noteworthy points
Life is like the ocean, it goes up and down.
Scripts bible for MySql
Be the Sun of your solar system.
Minium Workable Mvp Vimrc
”—————————————————————- “ 4. User interface “—————————————————————- “ Set X lines to the cursor when moving vertically set scrolloff=0
How to build unit/integration tests for Spring State Machine
Get busy living or get busy dying.
Magic after maven target spring-boot-run
Turn your wounds into wisdom
Bamboo pipeline deployment failure caused by Kubenetes Finalizer
Today a reader, tomorrow a leader.
Error in WSL in windows, command not found: sdk
Never stop learning, because life never stops teaching.
Could not write JSON: Value out of range. Value: “xxxxx” Radix:10
Life is really simple, but men insist on making it complicated.
Gemfire Geode Error on Peer or client version with ordinal xx not supported. Highest known version is 1.12.1 Client
Take the risk or lose the chance!
Password must not null in gemfire and geode, but I’ve assigned password in yaml properties file
Worries less, smile more!
One page to cover most commonly found errors for fat jar in SpringBoot
Kill time, or kiss time!
Awesome Shortcuts to boost productivity
One must learn by doing the thing; for though you think you know it, you have no certainty, until you try. —Sophocles
Core Java tips required an interview
Success is the sum of small efforts, repeated.
Tell me difference of truststore and keystore in short answer
Do what you say, say what you do.
GIT useful scripts or error solutions
Don’t wish for it, work for it.
Tell me difference of yarn install and npm install in short answer
Don’t find fault. Find a remedy.
Bible blog for most commonly found Gradle errors
People are smarter than you think. Give them a chance to prove themselves.
Pearls in Front end development
Be happy in front of people who don’t like you, it kills them.
Ruby from zero to hero
This is your life. Do what you love, and do it often.
Everything you’d know for Groovy interviews
Life is short. Don’t waste it with negative people who don’t appreciate you. Keep them in your heart but keep them out of your life.
To outstanding as professional MacBook pro user
The most effective way to do it, is to do it Homebrew The best practice is to run brew info before install new software. It will generally list what’s c...
Failed to install gem in Mac, incompatible architecture and missing psych
Burn your ego before it burns you.
IntelliJ Tips to boost your productivity 10 times
Don’t be afraid to make s splash.
Everything you’d know about state machine for interviews
Less expecting, more accepting.
Tips about algorithm resolving from Leetcode
Stay focused, believe that you can achieve at the highest level, surround yourself with others who believe in you and do not stray from your goals.
Solution center for Node errors
Fina a way. If there’s none, make one!
Triple your productivities by Visual studio code keyboard shortcuts
The sentence The quick brown fox jumps over the lazy dog uses every letter of the alphabet.
TypeScript noteworthy notes
The moment you start focusing on yourself, things start falling into place.
RXJS – reactive Programming like a hero
When love is real, it doesn’t lie, cheat, pretend or keep secrets.
Concurrency in Java
Little things make big things happens.
Linux Tips
Remember, some things have to end for better things to begin.
A taste of GraphQL
A good day starts with a good mindset!
A taste of GraphQL
A good day starts with a good mindset!
What’s inside magic in Spring Data JPA
A good day starts with a good mindset!
What’s inside magic in Spring Data JPA
A good day starts with a good mindset!
Why Spring turn a column name from camelNaming to snake_Naming
Don’t spend another year doing the same shit.
Some mistakes you’d avoid in java
With great power comes great responsibilities.
Untold stories for Jupiter, any differences JUnit 5 vs Junit 4
Don’t tell people your plans. Just show them your results!
Git commands you can show off for 100 years
Life is short, make a big splash!
FileNotFound Exception when loading data file in IntelliJ
Take time to do what makes your soul happy!
How to ace AWS certification just like play a game
Life isn’t about finding yourself. Life is about creating yourself.
Java Deep Notes
Java Deep Notes
Code to draw a Big H with all stars
Coding is everything! Code Now!
Code to draw a Big H with all stars
Coding is everything! Code Now!
Single vs Double precisions, float vs double data type
Leave nothing for tomorrow which can be done today. -Abraham Lincoln.
SkipTest-Not-Work-In-Multiple-models-project
Leave nothing for tomorrow which can be done today. -Abraham Lincoln.
SkipTest-Not-Work-In-Multiple-models-project
Leave nothing for tomorrow which can be done today. -Abraham Lincoln.
Maven error and solution on No such host is known
Don’t promis when you are hapy. Don’t reply when you’re angry and don’t decide when you’re sad Service keep on restarting If you spot service is restartin...
Maven error and solution on No such host is known
Don’t promis when you are hapy. Don’t reply when you’re angry and don’t decide when you’re sad
Gradle build stuck
Gradle build stuck, keep on running but never ending
2021
Save my eyes, let your cell phone to read screen content to you
Too much screen time
How logging system Bootstrapped in Spring Boot Application
Summary Following diagram demonstrated the process to bootstrap and use Logback for loggings in Spring Boot applciation.
SQLServer Error about This driver is not configured for integrated authentication
Symptoms When you are using integrated authentication (Kerberos connection) for MS SqlServer connection, there is one possible error :
How to copy files from resources folder in jar and save to a file
Why to extract resources from jar to local disk
Debug of SpringBoot run not working in IntelliJ
Normal approach to debug maven
How to watch specific kubenetes deployment by labels
How to watch specific kubenetes deployment by labels
Failed to talk to github.com from corporation network
Background It’s typical to get various network connection issues when you run commands within corporation network. For example, you’ll find diversed issues w...
Day-Day-Up-Java
More developer friendly Threa Sleep
How to user fire extinguisher
Summary As you know, staff and your safety is paramount. So what if emergency take place, such as fire in office, how to help yourself and your colleagues by...
Deep dive into ApplicationEvent in SpringBoot
Summary As you know, there are various event will be sent (multicast) when a specific story taken place.
2021-09-22-IT-Solutions-For-Remote-Learning
IT-Solutions-For-Remote-Learning.md
Deep dive into Kubernetes Client API
Summary To talk to K8s for getting data, there are few approaches. While K8s’ official Java library is the most widely used one. This blog will look into thi...
How to get CPU name, core, 64bit and speed in command line
Summary In windows operating system, if you want to get your CPU name, core, 64bit and speed in command line. Just follow below actions:
JetBrains/IntelliJ tips
Be a good person in real life, not in social media
Whitelabel Error Page
Summary Whitelabel Error Page is the default error page in Spring Boot web app. It provide a more user-friently error page whenever there are any issues when...
Google maps no photos reviews
Summary
谷歌地图里面照片的评论显示不出来
If you’d like to view solution in YouTube, check out at https://youtu.be/ICiwuqJ-yU8
Shall I still need booster even after I got dose 3?
The greatest wealth is health!
Debts in a nutshell
A debt security represents a debt owed by the issuer to an investor. Here, the investor acts as a lender to the issuer which may be a government, organisatio...
2020
How to process data from S3 download URL
S3 download URL As you know, AWS S3 object can be downloaded/processed by S3 download URL. I’m showing you two examples on how to process S3 Object by NIO f...
Debug Stuck IntelliJ
What happened to a debug job hanging in IntelliJ (IDEAS) IDE? You may find when you try to debug a class in Intellij but it stuck there and never proceed, e....
Awesome Kotlin
Difference with Scala Kotlin takes the best of Java and Scala, the response times are similar as working with Java natively, which is a considerable advantag...
Awesome tips for Chrome
Shortcuts & tips
JVM热身
此文是作者英文原文的翻译文章，英文原文在：http://todzhang.com/posts/2018-06-10-jvm-warm-up/
Awesome tips and shortcuts for Slack
Shortcuts for Slack
Awesome Reactive programming
Key points of Reactive Programming
Awesome Swift for iOS
Frame in Swift
Mock in kotlin
Argument Matching & Answers For example, you have mocked DOC with call(arg: Int): Intfunction. You want to return 1 if argument is greater than 5 and -1 ...
Mock in kotlin
Argument Matching & Answers For example, you have mocked DOC with call(arg: Int): Intfunction. You want to return 1 if argument is greater than 5 and -1 ...
Docker
Dockers Concepts
How to decode path parameters in All REST WebServices calls
How to decode path parameters in All REST WebServices calls
Curl
Linux Curl command
AOP
The concept of join points as matched by pointcut expressions is central to AOP, and Spring uses the AspectJ pointcut expression language by default.
Micrometer notes
As a general rule it should be possible to use the name as a pivot. Dimensions allow a particular named metric to be sliced to drill down and reason about th...
Pigeons in holes principle
# Pigeonhole principle
Awesome solutions for algorithm questions
你就会发现只要涉及递归的问题，都是 树的问题。
A Facial Recognition utility in a dozen of LOC
A Facial Recognition utility in a dozen of python LOC (Lines Of Code)
Awesome SSL certificates and HTTPS
What’s TLS TLS (Transport Layer Security) and its predecessor, SSL (Secure Sockets Layer), are security protocols designed to secure the communication betwee...
JVM warm up by Escape Analysis
Why JVM need warm up I don’t know how and why you get to this blog. But I know the key words in your mind are “warm” for JVM. As the name “warm up” suggested...
Java Concurrent Column 2
This is the second half about Java Concurrent of my blog
Java Concurrent
This blog is about noteworthy pivot points about Java Concurrent Framework Back to Java old days there were wait()/notify() which is error prone, while fr...
Algorithm notes from Leecode – 1
Algorithm Leetcode
2019
Conversations with God
Feelings is the language of the soul. If you want to know what’s true for you about something, look to how your’re feeling about.
Kafka In Spring
Enable Kafka listener annotated endpoints that are created under the covers by a AbstractListenerContainerFactory. To be used on Configuration classes as fol...
Terraform
Why Terraform
Kafka
Kafka
Mifid
FX Spot is not covered by the regulation, as it is not considered to be a financial instrument by ESMA, the European Union (EU) regulator. As FX is considere...
Foreign Exchange
currency pairs Direct ccy: means USD is part of currency pair Cross ccy: means ccy wihtout USD, so except NDF, the deal will be split to legs, both with...
2018
Seconds
nano seconds
Citrix receiver
Simple Binary Encoding (SBE)
Citrix receiver
“Cannot connect to remote desktop” with Citrix Receiver
Guice
A new type of Juice Put simply, Guice alleviates the need for factories and the use of new in your Java code. Think of Guice’s @Inject as the new new. You wi...
YAML
Key points All YAML files (regardless of their association with Ansible or not) can optionally begin with — and end with …. This is part of the YAML format a...
Distruptor
multithreading
Mockito
Feature
Protobuf
What are protocol buffers?
Sudo in a Nutshell
Sudo in a Nutshell Sudo (su “do”) allows a system administrator to give certain users (or groups of users) the ability to run some (or all) commands as root...
Zoo-keeper
ZK Motto the motto “ZooKeeper: Because Coordinating Distributed Systems is a Zoo.”
Presto DB
WHAT IS PRESTO?
Chronicle
Overview
Cucumber
Acceptance testing vs unit test It’s sometimes said that unit tests ensure you build the thing right, whereas acceptance tests ensure you build the right thi...
Scala
Scala String
akka framework of scala
philosophy The actor model adopts the philosophy that everything is an actor. This is similar to the everything is an object philosophy used by some object-o...
File Util in Apache Camel
FileUtil.class
Apache Camel
Camel’s message model In Camel, there are two abstractions for modeling messages, both of which we’ll cover in this section. org.apache.camel.Message—The ...
QuickFixJ
Settings
JXM
Exporting your beans to JMX The core class in Spring’s JMX framework is the MBeanExporter. This class is responsible for taking your Spring beans and registe...
Solace MQ
Solace PubSub+ It is a message broker that lets you establish event-driven interactions between applications and microservices across hybrid cloud environmen...
Apigee
App deployment, configuration management and orchestration - all from one system. Ansible is powerful IT automation that you can learn quickly.
Ansible
Ansible: What Is It Good For? Ansible is often described as a configuration management tool, and is typically mentioned in the same breath as Chef, Puppet, a...
flexbox
How Flexbox works — explained with big, colorful, animated gifs
Jboss tips
commands:
Locking and multithreading
Single Writer principle
KDB
KDB However kdb+ evaluates expressions right-to-left. There are no precedence rules. The reason commonly given for this behaviour is that it is a much simple...
Foreign Exchange
Foreign Exchange markets
Portactor
Better to use smart wait
Agile and SCRUM
Key concept In Scrum, a team is cross functional, meaning everyone is needed to take a feature from idea to implementation.
DevOps-Philosophy
:100:DevOps Model Defined
rxjs pipe in depth
https://stormforger.com/blog/2016/07/08/types-of-performance-testing/
How to setup nodejs to install package from intranet
Error of ‘ECONNRESET’ You may face error ECONNRESET from intranet, even appropriate proxy tools (e.g. cntlm) is running. The errors may looks like ```bash $ ...
Strategy-Of-Openshift-Releases
Release & Testing Strategy There are various methods for safely releasing changes to Production. Each team must select what is appropriate for their own ...
NodeJs Notes
commands to read files var lineReader = require(‘readline’).createInterface({ input: require(‘fs’).createReadStream(‘C:\dev\node\input\git_reset_files.tx...
Minium Viable Product
https://blog.leanstack.com/minimum-viable-product-mvp-7e280b0b9418
What is difference between declarations, providers and import in NgModule
What is difference between declarations, providers and import in NgModule
CORS :Cross-Origin Resource Sharing
Cross-Origin Request Sharing - CORS (A.K.A. Cross-Domain AJAX request) is an issue that most web developers might encounter, according to Same-Origin-Policy,...
ngrx
Why @Effects? In a simple ngrx/store project without ngrx/effects there is really no good place to put your async calls. Suppose a user clicks on a button or...
iOS programming
View A view is also a responder (UIView is a subclass of UIResponder). This means that a view is subject to user interactions, such as taps and swipes. Thus,...
2017
cloud computering
openshift vs openstack The shoft and direct answer is `OpenShift Origin can run on top of OpenStack. They are complementary projects that work well together....
cloud computering
Concepts Cloud computing is the on-demand demand delivery of compute database storage applications and other IT resources through a cloud services platform v...
Redux
whats @Effects You can almost think of your Effects as special kinds of reducer functions that are meant to be a place for you to put your async calls in suc...
reactive programing
The second advantage to a lazy subscription is that the observable doesn’t hold onto data by default. In the previous example, each event generated by the in...
common errors in NPM or node
code E503 code E503 when run npm install packages, e.g.
Container
The Docker project was responsible for popularizing container development in Linux systems. The original project defined a command and service (both named do...
promise vs observiable
The drawback of using Promises is that they’re unable to handle data sources that produce more than one value, like mouse movements or sequences of bytes in ...
Openshift tips
Commands bible
google analysis
How Page Value is calculated
JDK source
interface RandomAccess Marker interface used by List implementations to indicate that they support fast (generally constant time) random access. The primary ...
SSH SFTP
Secure FTP SFTP over FTP is the equivalant of HTTPS over HTTP, the security version
Setup WebSphere profiles and application in command line
Setup WebSphere profiles and application in command line
AWS Tips
After establishing a SSH session, you can install a default web server by executing sudo yum install httpd -y. To start the web server, type sudo service htt...
Oracle
ORA-12899: Value Too Large for Column
Spring notes
Spring Bean Life Cycle Callback Methods
Kindle notes
#《亿级流量网站架构核心技术》目录一览 TCP四层负载均衡 使用Hystrix实现隔离 基于Servlet3实现请求隔离 限流算法 令牌桶算法 漏桶算法 分布式限流 redis+lua实现 Nginx+Lua实现 使用sharding-jdbc分库分表 Disruptor+Redis...
Java JIT compiler
This is talking about Java JIT (Just-In-Time) compiler
Java Security Notes
Java Security well-behaved: programs should be prevent from consuming too much system resources
SeriableVersionUID
Noteworthy points about SeriableVersionUID in Java
R Language
s<-read.csv("C:/Users/xxx/dev/R/IRS/SHH_SCHISHG.csv") # aggregate s2<-table(s$Original.CP) s3<-as.data.frame(s2) # extract by Frequency ordered s3...
SSH and Cryptography
SFTP versus FTPS SS: Secure Shell An increasing number of our customers are looking to move away from standard FTP for transferring data, so we are ofte...
Eclipse notes
How do I remove a plug-in? Run Help > About Eclipse > Installation Details, select the software you no longer want and click Uninstall. (On Macintosh i...
Java JVM
Class loading subsystem
Maven-Notes
Maven philosophy “It is important to note that in the pom.xml file you specify the what and not the how. The pom.xml file can also serve as a documentatio...
Java New IO
Notes JDK 1.0 introduced rudimentary I/O facilities for accessing the file system (to create a directory, remove a file, or perform another task), accessi...
Network Protocols
Net Protocols
IT-Architect
SOA SOA is a set of design principles for building a suite of interoperable, flexible and reusable services based architecture. top-down and bottom-up a...
Algorithm
This page is about key points about Algorithm
Dead Lock
Concept
Java-Tricky-Tech-Questions.md
What is the difference between Serializable and Externalizable in Java? In earlier version of Java, reflection was very slow, and so serializaing large ob...
NavigableMap Misc
What is NavigableMap
Compare-In-Java
Concepts If you implement Comparable interface and override compareTo() method it must be consistent with equals() method i.e. for equal object by equals(...
Java Collections Misc
Difference between equals and deepEquals of Arrays in Java Arrays.equals() method does not compare recursively if an array contains another array on oth...
HashMap in JDK
Hashmap in JDK Some note worth points about hashmap Lookup process Step# 1: Quickly determine the bucket number in which this element may resid...
Java 8 Tips
This blog is listing key new features introduced in Java 8
Arbitrage vs Heading
What is the difference between arbitrage and hedging?
Java Enum Misc
Enum Misc
2016
Java GC notes
verbose:gc verbose:gc prints right after each gc collection and prints details about each generation memory details. Here is blog on how to read verbose gc
Hash Code Misc
contract of hashCode : Whenever it is invoked on the same object more than once during an execution of a Java application, the hashCode method must consis...
Apache Tips
Apache
Angulary Misc
Dependency Injection Angular doesn’t automatically know how you want to create instances of your services or the injector to create your service. You must co...
Random number in java
ThreadLocalRandom, SecureRandm, java.util.Random, java.math.Random
Java new features
JDK Versions JDK 1.5 in 2005 JDK 1.6 in 2006 JDK 1.7 in 2011 JDK 1.8 in 2014 Sun之前风光无限，但是在2010年1月27号被Oracle收购。 在被Oracle收购后对外承诺要回到每2年一个realse的节奏。但是20...
用10几行代码自己写个人脸识别程序
用10几行代码自己写个人脸识别程序
Eslastic Search
Eslastic Search
JSON lines
JSON lines
Python Scraphy
Python Scraphy
Simpler chronicle of CI(Continuous Integration) “乱弹系列”之持续集成工具
引言 有句话说有人的地方就有江湖，同样，有江湖的地方就有恩怨。在软件行业历史长河（虽然相对于其他行业来说，软件行业的历史实在太短了，但是确是充满了智慧的碰撞也是十分的精彩）中有一些恩怨情愁，分分合合的小故事，比如类似的有，从一套代码发展出来后面由于合同到期就分道扬镳，然后各自发展成独门产品的Sybase DB和微...
Head First Blockchina 3
Hyperledger Fabric for Mortals
【原创】深入浅出区块链系统：第二章
使用Solidity创建以太坊(Ethereum)智能合约(Smart Contract)
How to customize Sublime syntax highlights
Reference Sublime Scope Naming Syntax Guide
浅谈软件单元测试中的“断言” (assert)，从石器时代进步到黄金时代。
大家都知道，在软件测试特别是在单元测试时,必用的一个功能就是“断言”（Assert)，可能有些人觉得不就一个Assert语句，没啥花头，也有很多人用起来也是懵懵懂懂，认为只要是Assert开头的方法，拿过来就用。一个偶然的机会跟人聊到此功能，觉得还是有必要在此整理一下如何使用以及对“断言”的理解。希望可以帮助大家...
Head First Blockchina 1
深入浅出区块链系统：第一章. what you should know about blockchain
Kubernetes 与 Docker Swarm的对比
Kubernetes 和Docker Swarm 可能是使用最广泛的工具，用于在集群环境中部署容器。但是这两个工具还是有很大的差别。
漫谈开发设计中的一些‘原则’及’设计哲学’
在开发设计中有一些常用原则或者潜规则，根据笔者的经验，这里稍微总结一下最最常用的，以飨读者。
http methods
RFC origion http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1.2)
Spark-vs-Storm
The stark difference among Spark and Storm. Although both are claimed to process the streaming data in real time. But Spark processes it as micro-batches; wh...
微服务
可以想像一下，之前的传统应用系统，像是一个大办公室里面，有各个部门，销售部，采购部，财务部。办一件事情效率比较高。但是也有一些弊端，首先，各部门都在一个房间里。
unmodifiableList, unmodifiableSet,unmodifiableMap
What’s it Returns an unmodifiable view of the specified set. This method allows modules to provide users with “read-only” access to internal sets. Query ope...
kibana, view layer of elasticsearch
What’s Kibana kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on...
kibana, view layer of elasticsearch
What’s Kibana kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on...
Anatomy of ThreadLocal
Design philosophies
iConnect
UI HTML5, AngularJS, BootStrap, REST API, JSON Backend Hadoop core (HDFS), Hive, HBase, MapReduce, Oozie, Pig, Solr
Business Analysis
Purpose of BA 带来一些商业价值（收益） 解决业务痛点
Something about RESTful architect
REST API must be hypertext driver Roy’s interview
Data Structure
Binary Tree A binary tree is a tree in which no node can have more than two children. A property of a binary tree that is sometimes important is that th...
Useful bookmarks
eBooks list of various books Node.js
heavy load web application
Common solutions
tips in as400 IBM Emulator
Toggle crosshair
Mysql operator to extract JSON
“Be the change you wish to see in the world.” - Mahatma Gandhi
equity trading
Difference between mutal funds and hedge funds
SQL
Differences between not in, not exists , and left join with null
HTTPS/2
concepts
Github page commands notes
404 error for customized domain (such as godday) 404 There is not a GitHub Pages site here. Go to github master branch for gitpages site, manually add CN...
RenMinBi International
RQFII RQFII stands for Renminbi Qualified Foreign Institutional Investor. RQFII was introduced in 2011 to allow qualified foreign institutional investors to ...
JavaScript tips
includes() vs some()
Docker Errors and Fixes
Docker Errors
Load Balancing
Concepts LVS means Linux Virtual Server, which is one Linux built-in component.
Python
(‘—–Unexpected error:’, <type ‘exceptions.TypeError’>) datetime.datetime.now()
Storage Management
RAID RAID is Reductant Array Independent Disk,
CI and CD
Concepts
XA Transactions in 2PC
Description
Setup Git in Mint Linux
How to setup Git in Mint Linux =================================================
Database sharding
DB sharding in YHD
Microservices vs. SOA
Microservice Services are organized around capabilities, e.g., user interface front-end, recommendation, logistics, billing, etc. Services are small in ...
Java Class Loader
Codecache The maximum size of the code cache is set via the -XX:ReservedCodeCacheSize=N flag (where N is the default just mentioned for the particular com...