CentOS 6 Install netcat

Netcat is one utility in NMap toolset, it maybe named as nc, ncat or netcat on different Linux distributions. (It’s ncat on CentOS 6).

On CentOS 6 netcat is not installed by default, to install it we can run following command

 

Then run following command to check netcat is installed correctly

 

Cannot install Docker on CentOS 6

Installing docker-ce on CentOS 6.9 got following error

 

To install docker on CentOS 6, run the following command

 

Then run

Its output is

Docker version 1.7.1, build 786b29d/1.7.1

 

We can see docker is installed successfully.

Spring Cloud Zuul Ribbon custom rule not working

I need a custom Rule which can route requests to different service according to weight option (actually I want to deploy an A/B test service and only small percent of user will hit it), the code is like following

 

And following is application.yml configuration file

 

Adding breakpoints at constructor of ZoneAwareLoadBalancer and DynamicServerListLoadBalancer, then access the Zuul service via browser and I found ZoneAwareLoadBalancer constructor breakpoint is reached first.

Inspect the value clientConfig->properties->NFLoadBalancerClassName, its value is still “com.netflix.loadbalancer.ZoneAwareLoadBalancer”, and the passed in rule is not my configured rule as well.

This means my configuration didn’t work.

 

Even I changed the class name to an invalid value, there is no error

 

In Spring Cloud documentation, under Customizing the Ribbon Client using properties section, it’s saying

Starting with version 1.2.0, Spring Cloud Netflix now supports customizing Ribbon clients using properties to be compatible with the Ribbon documentation.

Checking maven pom file I found it’s using 1.1.0

 

Changing them to 1.2.0 and run again, following exception is thrown

 

It’s because NFLoadBalancerRuleClassName is aaa, changing it to a valid name and try it again, it works now

 

 

Intellij IDEA plugin development get selected text

To get selected text in a Intellij Platform Plugin project, we can use following code

 

The first line is used to get current editor object

 

Then call getSelectionModel and getSelectedText method to get selected text

 

The last two lines is to to some operation for selected text, it’s unrelated to our goal.

 

 

HBase get column qualifiers by column family

To get column qualifiers by column family we can use following code

Above code will outputs all column qualifer names of family map “basicInfo” in table “merchants”.

 

And following code shows different ways to retrieve HBase cell value

 

  • Get Cell by Family Map and Column Qualifier

This line of code retrives cell value by family map “merchantNo” and column qualifier “string”

 

  • Get All Cells by Faimily Map

result.getFamilyMap(Bytes.toBytes("merchantNo"))  will return column family map with name “merchantNo”, it contains column qualifier name as key and cell value as value.

Above code will iterate the column family map, and outputs each column qualifier and its corresponding cell value.

mongodb InternalError: failed to create service entry worker thread

Following is content from /var/log/mongodb/mongod.log:

2017-12-18T01:21:41.953+0800 W EXECUTOR [conn4444] Terminating session due to error: InternalError: failed to create service entry worker thread
2017-12-18T01:21:41.953+0800 I NETWORK [listener] end connection 127.0.0.1:36868 (1001 connections now open)
2017-12-18T01:21:41.953+0800 I NETWORK [listener] connection accepted from 127.0.0.1:36870 #4445 (1002 connections now open)
2017-12-18T01:21:41.954+0800 I – [listener] pthread_create failed: Resource temporarily unavailable
2017-12-18T01:21:41.954+0800 W EXECUTOR [conn4445] Terminating session due to error: InternalError: failed to create service entry worker thread
2017-12-18T01:21:41.954+0800 I NETWORK [listener] end connection 127.0.0.1:36870 (1001 connections now open)
2017-12-18T01:21:41.979+0800 I NETWORK [listener] connection accepted from 127.0.0.1:36872 #4446 (1002 connections now open)
2017-12-18T01:21:41.979+0800 I – [listener] pthread_create failed: Resource temporarily unavailable
2017-12-18T01:21:41.979+0800 W EXECUTOR [conn4446] Terminating session due to error: InternalError: failed to create service entry worker thread
2017-12-18T01:21:41.979+0800 I NETWORK [listener] end connection 127.0.0.1:36872 (1001 connections now open)

 

This problem occured after web request (which will create new MongoClient instance) started for a while, it’s not happened immediately, so it’s likely caused by the connection limit is reached.

Through the log we can find the connection count limit is about 1000.

 

Every time I open Mongo Console, I got startup warnings

MongoDB shell version v3.6.0
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.0
Server has startup warnings:
2017-12-19T16:29:45.510+0800 I STORAGE [initandlisten]
2017-12-19T16:29:45.510+0800 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2017-12-19T16:29:45.510+0800 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
2017-12-19T16:29:46.006+0800 I CONTROL [initandlisten]
2017-12-19T16:29:46.006+0800 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-12-19T16:29:46.006+0800 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2017-12-19T16:29:46.006+0800 I CONTROL [initandlisten]
2017-12-19T16:29:46.008+0800 I CONTROL [initandlisten]
2017-12-19T16:29:46.008+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is ‘always’.
2017-12-19T16:29:46.008+0800 I CONTROL [initandlisten] ** We suggest setting it to ‘never’
2017-12-19T16:29:46.008+0800 I CONTROL [initandlisten]
2017-12-19T16:29:46.008+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is ‘always’.
2017-12-19T16:29:46.008+0800 I CONTROL [initandlisten] ** We suggest setting it to ‘never’
2017-12-19T16:29:46.008+0800 I CONTROL [initandlisten]
2017-12-19T16:29:46.008+0800 I CONTROL [initandlisten] ** WARNING: soft rlimits too low. rlimits set to 1024 processes, 64000 files. Number of processes should be at least 32000 : 0.5 times number of files.
2017-12-19T16:29:46.008+0800 I CONTROL [initandlisten]
We can see the connection count limit is actually 1000.

 

So to fix this issue, either is to increase the connecton count limit, or to close the created MongoClient explicitly in code.
In my program every request a new MongoClient is created, I didn’t explicitly close it after using it. The solution is calling close method of MongoClient or using singleton MongoClient (the MongoClient library recommended this way for using connection pooling).

By using this method, the problem is solved.

Python TypeError: Error when calling the metaclass bases

When running the code, I got following error:

TypeError: Error when calling the metaclass bases
module.__init__() takes at most 2 arguments (3 given)

 

The source code is like following:

AboutWindow.py

 

GitFtpWindow.py

 

This is because in AboutWindow.py file, the GitFtpWindow identifier is a package, but not a class.

To fix this problem, we need modify AboutWindow.py file like following:

 

 

Java HTML Encoding (HTML Entities)

Some XSS attacks can be prevented by using HTML Encoding.

HTML encoding function is built into many languages, In .NET WebUtility.HtmlEncode  can do it, in PHP we can use htmlentites  function, in Python cgi.escape  can be used.

But there is no built-in function to do HTML Encode (or HTML Entities) in Java.

We can use Apache Commons Lang library to do this work.

 

Above code will output following result using HTML Encoding

<>

 

Note that not all XSS attacks can be prevented by HTML encoding (https://stackoverflow.com/questions/53728/will-html-encoding-prevent-all-kinds-of-xss-attacks).

Apache POI Sheet.getPhysicalNumberOfRows()

In the past I use following code to display first cell string of all rows.

But above code will be incorrect if there is empty row.

 

Following is explanation from POI official documentation

Sheet.getPhysicalNumberOfRows()

Returns the number of physically defined rows (NOT the number of rows in the sheet)

This method will ignore empty rows, e.g. if a sheet last row is at row 7, but second row is empty, getPhysicalNumberOfRows()  will return 6.

 

Solution

To get row count of a sheet (no matter the row is empty or not), we should use getLastRowNum()  method.

So above code can be changed to

Because getLastRowNum()  method returns 0-based row index, so we use i<=sheet.getLastRowNum()  as the loop condition.

Dom4j Writing File Not Working

Dom4j is a XML processing library which supports XPath, DOM, SAX,  JAXP. In this post we will see a problem that dom4j could not write document to file.

 

Following XML is sample data which we will save to file using dom4j.

To generate above XML and write it to file, following code will be used

Above code may generate an empty file with no content.

 

This is because the content is stored in buffer and not really written to file when calling write  method. To write to file, we should call flush  method or close  method.

So above code should be changed to

Or