I/O Stats Web Interface

From Wiki

Revision as of 20:46, 26 July 2011 by Admin (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Contents

I/O Stats Overview

Functionality

As you know most programs/scripts they read, compute and write data during their execution. Usually the processes of reading and writing data is quite simple. You open a file from the disk or you save a file to the disk. These are the so called Input/Output operations also known as I/O. For the kernel(the OS) however, the process is more complicated. Thus in order to simplify things the I/O statistics in Hive are exported from the kernel are collected at the program level.

NB: Note that there is difference between the actual Disk I/O(the I/O operations executed only on the local disk, raid, san,

nas, or other storage you are using) and the total I/O of a program. Each time a program executes a call to the system function read() the kernel collects the read bytes and stores them in the stats for this process. The situation is the same with writes. The distinction comes because a program can read a single file from the disk and store it in memory once, then reread the same file from the memory multiple times. So the total I/O operations for a program are not all disk related I/O operations. This is why we collect separately the total and the disk I/O operations of the process.

Since Hive knows the ID of each process it runs, it can easily collect the I/O usage information from it. What it does is to check the I/O usage every 0.2 seconds while the process work and when the process finish store the collected data.

I/O Usage Data Collection

All I/O usage is gathered by a daemon called [Multistatsd]. This daemon is responsible for storing the statistical information for CPU time, Memory usage, I/O usage and process executions. It acts as a cache for the statistics because it stores the data in its memory until the first minute of the next hour. When it reaches the first minute it uploads the accumulated data into a PostgreSQL data base. The design of the software was done in this way because we wanted efficiency and performance.

All I/O operations are recorded in bytes. However since the values are really high when represented in just bytes they are all represented in MBytes under the Hive Web Interface for I/O Stats.

I/O Stats Web Interface

I/O Stats - Admin Portal Web Interface

This section will be available as soon as Memory statistics are available in the Central Portal.

I/O Stats - Local Web Interface

The I/O statistics interface can be reached via the 1H Local Interface for the server. The button 'I/O Stats' is located on the left side panel(just bellow the 'Memory stats' button).

Upon accessing the Interface, the the general server information will be loaded. This screen shows you the I/O usage of all users on the server.

NB: Note that you will not see the total I/O usage, only the I/O usage for chrooted programs executed through Hive.

The I/O interface is a bit different from the other interfaces for 1H Hive Statistics. The difference is that you have two separate statistics for the Server and for Users. You will be able to check in separate screens the disk only reads/writes and the total reads/writes for both Server and for User.

The 4 different statistics you will be able to check are:

Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox