I tried using real-time access visualization tool "Hummingbird"

If you have your own web page or website, you would like to know the trend of the user you are viewing in real time. In GIGAZINE also, it can not display access in real time, so the tool "HummingbirdI tried using it.

Details are as below.
Mnutt / hummingbird @ GitHub

A movie that is actually used and the bar graph is displayed on the browser is here. The horizontal axis shows time and the vertical axis shows the number of accesses.HTML5Is a technologyWeb Socket, It is possible to display access information at very short intervals of 50 milliseconds.

YouTube - Real-time access visualization tool "Hummingbird"


MIT licenseIn Hummingbird distributed in Google ChromeJavaScriptengineV8Was usedEvent driven typeIt is a serverNode.jsWhenKey value storeWhenRelational databaseIt is said to be in the middle ofDocument Oriented DatabaseIsMongoDBIt is made using.

In addition, Google Chrome or Safari corresponding to WebSocket is required for display on the browser.

■ Introduction method at CentOS
1.First, install MongoDB. Create a file called 10gen.repo with the following contents in /etc/yum.repos.d.

[10 gen]
Name = 10 gen Repository
baseurl=http://downloads.mongodb.org/distros/centos/5.4/os/i386/
Gpgcheck = 0

Then install the package and start the service as follows.

Yum --enablerepo = 10gen install mongo - stable - server
Service mongod start

2.Next, install Node.js.

wget http://nodejs.org/dist/node-v0.1.100.tar.gz
Tar zxvf node-v0.1.100.tar.gz
Cd node-v0.1.100
./configure
Make
Make install

3.GitGet Hummingbird at.

Git clone git: //github.com/mnutt/hummingbird
Cd hummingbird
Git submodule update - init
Cd deps / express
Git submodule update - init

Four.Rename the configuration file app.json.sample in config directory to app.json. Note that app.json.sample is a text file like the following.

{
"Name": "Hummingbird",

"Monitor_port": 8888,
"Tracking_port": 8000,
"Websocket_port": 8080,

"Username": "admin",
"Password": "change_this",

"Capistrano": {
"Repository": "git: //github.com/mnutt/hummingbird.git",
"Hummingbird_host": "hummingbird.your-host.com"
}
}

Five."No method replace" errorIn order to deal with hummingbird / deps / express / lib / support / ejs / lib / ejs.js change as follows.

--- ejs.js.orig 2010-07-05 12: 19: 02.000000000 + 0900
+++ ejs.js 2010-07-05 12: 19: 14.000000000 + 0900
@@ - 11, 7 + 11, 7 @ @
New Function ("locals",
"Var p = [], print = function () {p.push.apply (p, arguments);};" +
"With (locals) {p.push ('" +
- str
+ Str.toString ()
.replace (/ [\ r \ t \ n] / g, "")
.split("<%").join("\t")
.replace (/ ((^ |%>) [^ \ t] *) '/ g, "$ 1 \ r")

6."No method queryString.parseQuery" errorChange hummingbird / deps / express / lib / express / plugins / body - decoder.js to deal with as follows.

--- body-decoder.js.orig 2010-07-09 13: 49: 32.000000000 + 0900
+++ body-decoder.js 2010-07-09 13: 49: 54.000000000 + 0900
@@ -2 -2,7 +22,7 @@
Var request = event.request
If (request.header ('Content-Type') & amp; Request.header ('Content-Type'). Includes ('application / x-www-form-urlencoded'))
- request.params.post = queryString.parseQuery (request.body)
+ Request.params.post = queryString.parse (request.body)
}
}
})

■ Execution
1.Run Hummingbird on the local machine.

Node server.js & amp;> log / hummingbird.log &
Node monitor.js & amp;> log / hummingbird_monitor.log & amp;>

2.ブラウザから「http://The machine on which Hummingbird was installed: 8888 / ", the following screen will be displayed. Enter" admin "in" USERNAME "and" edit_this "in" PASSWORD "and click" Login ".


3.The following access information display page is displayed. Access count information flows from left to right. The section that is slightly brighter gray on the graph is one second.


Four.別のタブを開いて「http://The machine on which Hummingbird was installed: 8000 / "will be accessed. A black dot with an image size of 1 x 1 used for tracking is displayed. Repeat reloading on this tab. For Chrome, hold down Ctrl and press R for 1 second.


Five.When you look at the access information display page again, a vertical bar indicating that you have accessed is displayed. As you can see from the screen, only about 4 seconds of content is displayed, so be careful. The color changes depending on the scale of the vertical axis.


■ Outline of operation
動作概要を示したものが下の図です。アクセスを監視したサイトのHTMLファイルにトラッキング用のタグ, you can see that Hummingbird got access when viewing this page.Hummingbird has its access information and browses it on the access information page as real time information can do.


■ Packet Dump
To see the state of access using WebSocket between browser and serverTcpick: a tcp stream sniffer, tracker and capturerI looked at the contents of the packet using. The commands used are as follows.

Tcpick - t - h - yU - i lo 'port 8080'

The result is as follows. This time we tested on the local machine, so 127.0.0.1: 42020 is the browser side, 127.0.0.1: http - alt is the WebSocket side. In addition, the output has been formatted for easy viewing, and the packet size 0 is actually displayed in the output, but it is omitted here.

Starting tcpick 0.2.1 at 2010-07-06 16: 20 JST
Timeout for connections is 600
Tcpick: listening on lo
Setting filter: "port 8080"
16: 20: 34.826348 127.0.0.1: 42020 S> 127.0.0.1: http - alt (0)
16: 20: 34.826949 1 SYN - SENT 127.0.0.1: 42020> 127.0.0.1: http - alt
16: 20: 34.827175 127.0.0.1: http - alt AS> 127.0.0.1: 42020 (0)
16: 20: 34.827412 1 SYN-RECEIVED 127.0.0.1: 42020> 127.0.0.1: http-alt
16: 20: 34.828004 1 ESTABLISHED 127.0.0.1: 42020> 127.0.0.1: http - alt
16: 20: 34.828541 127.0.0.1: 42020 AP> 127.0.0.1: http - alt (138)
GET / HTTP / 1.1
Upgrade: WebSocket
Connection: Upgrade
Host: 127.0.0.1: 8080
Origin: http://127.0.0.1:8888
Cookie: not_secret = admin

16: 20: 34.830126 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (166) HTTP / 1.1 101 Web Socket Protocol Handshake
Upgrade: WebSocket
Connection: Upgrade
WebSocket-Origin: http://127.0.0.1:8888
WebSocket-Location: ws: //127.0.0.1: 8080 /
16: 20: 34.831182 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (4)

16:20:34.847295 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 34.847844 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (24) {"total": 0, "cartAdds": 0}
16:20:34.848290 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)
16:20:34.899853 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 34.900962 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (24) {"total": 0, "cartAdds": 0}
16:20:34.901875 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)
16:20:34.943358 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 34.944887 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (24) {"total": 0, "cartAdds": 0}
16:20:34.945713 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)
16:20:34.995478 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 34.996123 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (24) {"total": 0, "cartAdds": 0}
16:20:34.996935 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)
16:20:35.043314 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 35.044116 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (24) {"total": 0, "cartAdds": 0}
16:20:35.044900 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)
16:20:35.096556 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 35.096980 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (24) {"total": 0, "cartAdds": 0}
16:20:35.097295 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)
16:20:35.143296 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 35.143700 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (24) {"total": 2, "cartAdds": 0}
16:20:35.144029 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)
16:20:35.195554 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 35.196201 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (24) {"total": 3, "cartAdds": 0}
16:20:35.196581 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)
16:20:35.244891 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 35.245252 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (24) {"total": 3, "cartAdds": 0}
16:20:35.245481 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)
16:20:35.251985 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 35.252324 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (12) {"sales": {}}
16:20:35.252773 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)
16:20:35.295527 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 35.296066 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (24) {"total": 6, "cartAdds": 0}
16:20:35.296353 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)

(Abbreviation)

16:20:39.443752 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 39.444226 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (24) {"total": 2, "cartAdds": 0}
16:20:39.444561 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)
16:20:39.495396 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1) <00>
16: 20: 39.496518 127.0.0.1: http - alt AP> 127.0.0.1: 42020 (24) {"total": 1, "cartAdds": 0}
16:20:39.496986 127.0.0.1:http-alt AP > 127.0.0.1:42020 (1)
16: 20: 39.499702 127.0.0.1: 42020 AF> 127.0.0.1: http - alt (0)
16: 20: 39.500109 1 FIN - WAIT - 1 127.0.0.1: 42020> 127.0.0.1: http - alt
16: 20: 39.539440 127.0.0.1: http - alt A> 127.0.0.1: 42020 (0)
16: 20: 39.539860 1 FIN - WAIT - 2 127.0.0.1: 42020> 127.0.0.1: http - alt

460 packets captured
1 tcp sessions detected

From the output result, you can see that the connection is established from the WebSocket side to the browser side immediately after establishing a connection from the browser side to the WebSocket side.

Also, the data transmitted by HummingBird is in accordance with the protocol of WebSocket (0x00 at the start of the data frame and 0xFF at the end, in a format including the text of UTF-8 in the meantime), and transmits the data in the JSON format You see that it is.

HummingBird is a pre-alpha version, and tabs are displayed as follows during browsing on Chrome in your environment.


In addition, some modifications are necessary at the time of installation, and it is still developing but it is expected to be expected from now on.