Kamajii GUI

Kamajii's GUI is the preferred way to update the configuration files that Kamajii uses to monitor web sites. This document assumes that you have already installed Kamajii.

Running the GUI

To run the GUI, enter the directory in which you have installed Kamajii and run "kam_conf.pl".

System Screen

The system screen is the first thing that Kamajii's GUI displays when it comes up:

This screen allows you to configure some general properties of Kamajii:

Cycle Time (seconds)
In each of its cycles, Kamajii checks all the web sites that it is configured to monitor and processes their new information. The amount of time that Kamajii should wait between cycles is configured in the Cycle Time parameter.

Sites Folder
The path to a directory in which Kamajii's site configuration is saved. The path can be relative to Kamajii's root directory.

Output Destinations
The Output Destinations list all the possible ways in which Kamajii can send alerts about the sites it monitors.
Output destinations are simply programs that Kamajii can run that can handle its output. When an output destination is double-clicked, its configuration opens in the right panel. In it, you can configure the command line that is run to send a message.

You can add new destinations by simply clicking on the + icon. The configuration for the new destination will open.

Output Destination Configuration

In the example above, a new output destination called "readit" has just been defined. To complete the definition, you would have to fill the three fields:

Command Line: What to run in order to send the message. The command line may use variables, which are surrounded by dollar signs. The $message$ variable is always defined - this is the place in the command line where the message to be sent will be written. Additionally, if multiple users are using the same Kamajii process, you can configure some variables which the user will be able to define in a different screen. These variables are configured by simply writing their names in the command line.

In the "readit" example, you could write a command line that runs a (hypothetical) text-to-speech engine that converts the message to text and reads it out loud: text2speech -text $message$ -voice $male_or_female$
This command line allows your users to decide if they prefer a male or a female voice reading their messages for them.

Maximum Message Length: Some output destinations, such as SMS, have a maximal number of characters that can be sent in a single message.

Separator: When multiple messages need to be sent to the same user at the same time, Kamajii combines them into one message. The string in the "Separator" field defines the separator that will be written between every two messages.

The default output destinations that Kamajii installs with are:

e-mail: Sends a notification via e-mail. You must configure the command line with your mail-server and from-address. If your mail server requires authentication, configure also your username and password on the command line.
SMS: Sends a notification via SMS, using Nadav Har'El's sendsms script. This script sends SMS to cell-phones of Israeli providers. If you are from another country you can search for a similar script that works for you and replace the command line.
stdout: Prints the notification to the standard output.

Saving
When you finish editing your configuration, save it by selecting "Save" from the file menu.

Adding Users

Kamajii can monitor sites for multiple users in the same process. Each user has his or her own password and the configurations are encrypted, so users cannot read each others' configurations.

To add a new user to Kamajii, choose File->New->User from the menu. A popup will open and prompt you to enter the user's name, the path for the user's configuration and a password. Once you enter these, the user's configuration will be created.

User Screen

The user screen is where you handle the user-specific configuration. This is where you tell Kamajii which sites you want it to monitor for you. To access the user screen, choose your name from the Configuration->Users menu. You will be prompted for a password, and a screen similar to the following one will open:

User Screen

Messages
On the left of the screen you will see a list of the output destinations that are configured for the system. You can select the ones that you want to have enabled for you by clicking on the box to the left of their names.

After choosing every destination in the list, the screen looks like this:

All the destinations that you can use to send messages are highlighted. You can now click on the red links in order to configure your information. This actually configures the user-specific parameters that were defined in the command line of the output destination in the user screen.

The e-mail configuration requires one parameter - your e-mail address. This is the address that Kamajii will send alerts to.
The SMS configuration requires several parameters for the sendsms script (this applies only to Israeli users):
- If you want your messages to be sent to a Cellcom or Pelephone phone:
  - If you don't already have one, create a login at ICQ.
  - In Kamajii's GUI, fill "icq" in the "company" field.
  - Fill either your e-mail address or your ICQ number in the username field.
  - Fill your password at ICQ in the password field.
- If you want your messages to be sent to an Orange phone:
  - If you don't already have one, create a login at Orange's site.
  - In Kamajii's GUI, fill "orange" in the "company" field.
  - Fill your login at Orange's site in the username field.
  - Fill your password at Orange's site in the password field.
  - The following important note is copied from the sendsms manual page. Please read it if you plan to use Kamajii to send SMS messages to Orange phones:

An important note regarding Orange: For this provider's official
SMS sending site, you're required to own their phone to complete the
registration for an online SMS sending account. This means two things:

First, if you do not own a phone from Orange you cannot
send SMSs to Orange phones.

The second issue with the official site is that since Orange can "prove" who
sent a message and connect it to your phone bill, they may decide some
time in the future to start charging for these messages. In fact, Orange
charged for this service for a few weeks (10 agorot per message) before
they decided to make the first 20 messages per day free. Sendsms tries
to verify that sending a message is still free (by looking at the text
of the SMS sending pages), but NO GUARANTEE IS MADE by me that your phone
bill does not get charged for the messages you send with sendsms. I
repeat: Orange may charge your phone bill for SMSs you send and sendsms
cannot and will not guarantee that this never happens.

Kamajii uses sendsms to send SMS messages so we do not guarantee that your phone bill does not get charged for the messages Kamajii sends you.

If you want your messages to be sent to an Amigo phone:
- If you don't already have one, create a login at Amigo's site.
- In Kamajii's GUI, fill "amigo in the "company" field.
- Fill your login at Amigo's site in the username field.
- Fill your password at Amigo's site in the password field

Sites
On the right of the user screen is the list of sites that are defined in your installation of Kamajii. In the installation that is shown in the picture, the sites Grades, Quotes, News and Sports have been defined.
Clicking on the box to the right of a site marks that site to be monitored for you by Kamajii. Sites often require user-specific configuration. This configuration can be accessed by clicking on the site's name. You can also configure, for each site, which of the types of messages that you have selected (on the left) are used. This is done by clicking the site's "[messages]" link.

The user screen with the "Grades" site selected:

Choosing "Grades" opens a popup that asks you to fill site-specific information:

Grades Site

Clicking on the [messages] link to the right of "Grades" opens the following popup:

Site messages configuration

Saving
When you finish editing your configuration, save it by selecting "Save" from the file menu.

Default Sites

Kamajii's sites are highly configurable, and you are encouraged to add sites that you find useful using the Site Screen. However, Kamajii installs with a few site configuration files:

FIBI
FIBI is the First International Bank of Israel. Their site is at http://www.fibi.co.il/. If you have an account at FIBI and registered at your bank for a username and password that enable you to log-in and see your bank account online, you can use three configurations to monitor your bank account without opening your browser. In all three configurations, fill your FIBI username in the "username" field and your FIBI password in the "password" field.

fibi-Balance-Changes: Notifies you whenever your balance changes.
fibi-Limits: Notifies you whenever your balance goes below a configurable limit and whenever it goes above another configurable limit. You need to configure "Min_Yitra" to the minimal balance and "Max_Yitra" to the maximal one. For example, if you configure Min_Yitra to 0 and Max_Yitra to 10000, you'll get a notification when your bank account is in overdraft, and also when your balance is very high, which probably means you should invest your money.
fibi-Daf_Heshbon: Notifies you whenever an operation is done on your account with the details of this operation. For example: "New operation in fibi. [ ] + / - [ 100 ]: כספומט", or "New operation in fibi. [ 1000000 ] + / - [ ] : לוטו".

Haaretz
Notifies you on the current events from Haaretz news.

Whatsup
Notifies you on linux and open-source news from the Israeli site Whatsup.

TAU
If you are a student at the Tel Aviv University, use this to get notifications whenever you receive a new grade. You will need to fill in your ID number in the "id" field and your password for http://www.ims.tau.ac.il/tal/ in the "password" field.

Quotes
Notifies you on changes in a stock, according to Yahoo! Finance. You will need to fill in the stock's symbol (for example, for Yahoo! fill in "YHOO"), the bottom threshold for the stock (you will be notified when the stock goes under this value) in Minimum_Trade, and the upper threshold in Maximum_Trade.

Galgalatz
Traffic updates from Galgalatz.

Site Screen

This screen is used to define new sites for Kamajii to monitor. It is more complicated than the other screens, so it is recommended that you familiarize yourself with Kamajii using the default sites before delving into creating new ones.

Still here? Very well then..

A site definition is divided to the parameter definitions and the page definitions. Parameters are inputs that are different for different users. For example, password or username. Everyone who uses the site you define will be required to fill in these parameters. The pages define the flow of the site - which pages Kamajii needs to request and what it needs to do with them.

Parameter Definitions
Parameter Screen

The image above shows the list of parameters for the site fibi-Balance-Changes. Double-clicking on a parameter will open its configuration details:

Min_Yitra parameter

As you can see, there are two types of parameters:

User-Defined parameters (as seen in the image above): The user of the site will be requested to enter these parameters. You can restrict what she enters by setting "length" to the length of this data and/or "Regular expression" to a perl-style regular expression that the data must match.
Non User-Defined parameters: Parameters that you define the value of when configuring the site. You must fill in the "value" field, and the value of the parameter will always be this value. These parameters can be useful if you have a long expression that is used several times in the rest of the site configuration. For example, you could define a parameter called "pi" with the value "3.14159265358".

Page Definitions

Clicking on the "Pages" tab brings you to the Page Definition screen:

Pages

Here you can see the pages of the fibi-Balance-Changes. Kamajii will request the pages in the order they are written. The buttons above the list allow you to add, rename and delete pages, and to move them up or down in the list. Double clicking on a page name shows you the page definition. For example, double-clicking on "balance_page" will show the following:

balance_page: Request definition

As you can see, a new set of tabs has opened to the right of the screen. Each tab allows you to define a different set of properties for the page.

Before entering all the possible configuration options, the following general rule works in pretty much every field: Entering the name of a parameter that is already defined, surrounded by $ signs, into a string, will replace the parameter's name with its value. For example, if you define a parameter "name" with the value "Herman", the string "My name is $name$" will be replaced with "My name is Herman". In the next sections you will see several examples of where this behavior can be useful.

Request

Method: HTTP method to use when requesting. In general, if the page is the result of the posting of a form that has POST as its method, set Method to "POST" and otherwise set it to "GET".
URL: The URL to request. The URL sometimes contains parameters. In case of the URL in the picture, "$snif$" will be replaced with the "snif" parameter that was defined in a previous page.
HTTP Version: The HTTP version to use when sending the request. If the site you are trying to monitor is using a very old web server, you might need to change it to "1.0".
HTTP Headers: Kamajii creates all the HTTP headers that it needs when sending the request (for example, it handles cookies automatically). Some sites may require specific headers that Kamajii does not send by default. These headers can be defined here. For example, if a site works only with a specific user-agent (i.e. a specific browser), you can override the User-Agent header with this browser's details as the site expects them.
HTTP Request Content: The request content is usually needed when the page is the result of the posting of a form that has POST as its method. In this case, it contains all the posted parameters with their values: name1=value1&name2=value2... To find the names of the parameters, look at the HTML source of the form. Every <INPUT> tag contains the name of a parameter.

Note: Whenever a "URL" or an "HTTP Request Content" item contains a reference to a predefined list parameter (of size N), the current HTTP request is sent and processed N times, each time using a different element from the list. In case more than one list is referenced in "URL" and "HTTP Request Content", all these lists, which are assumed to be of the same size, are scanned that way.

Reply Status Code: The status code that the web server is expected to return afterit receives the request. The default is 200, which means "OK" in HTTP.

Error Handling

Send Alert: When this checkbox is checked, errors on this page are sent to the users of the page using the output destinations they have configured for the site. When it is unchecked, the errors are only written to the standard error.
Error Page: Write here the name of a page that Kamajii should go to when an error occurs on the current page. This is normally used for sites in which you need to log off. The error page must come after the current page in the list of pages.

Content Parameters
This section lists the parameters that can be defined based on the content of the request.
The following types of parameters are defined:

Regex

Regex parameters search for a regular expression on the contents of the reply, and save its results.

Regular Expression: Perl-style regular expression to search for.
Appearance: can be either the number of the appearance, or "list" (the parameter is set to the list of appearances) or "uniq_list" (the parameter is set to the list of unique appearances).
Format: Each match of the regular expression with the reply is formatted according to what is written in this field. The format can contain characters, variables that were defined previously, and the variables $1$, $2$, etc', which are the strings that matched bracket #1, bracket #2, etc.
Save to DB as: Every parameter can be saved to the database, in order to use it in the future to check for differences. If you want the parameter to be saved to the database, define its name in the database in this field. Note: the database is encrypted with the 3DES algorithm, so as long as you keep your password secret, it is ok to save even sensitive data in the database.

Table

Table parameters search for HTTP tables in the contents of the reply and return some of their rows and columns.

Number of table in HTML: If there are multiple tables in the reply, this defines the index of the table you are interested in. Note that the count starts with 0.
Ranges of Columns: For example the range: "2,3,7-9" means that columns 2, 3, 7, 8 and 9 will be read. Note that the count starts with 0.
Ranges of Rows: For example the range: "2,3,7-9" means that rows 2, 3, 7, 8 and 9 will be read. Note that the count starts with 0.
Save to DB as: Same as in Regex.

DB parameters load the values of parameters that were saved to the database in the previous run.

Name of Database Variable: The name that you wrote for the parameter that you saved in the field "Save to DB as".
Save to DB as: If you need to save a longer history, you can save this variable back to the database by stating a new name for it here.

Differences
Differences are your way to check for changes in the page, which will trigger alerts.
This is the Differences tab of fibi-Balance-Changes:

The list on the right of the screen is a list of the differences to be checked. New ones can be added by clicking +, existing ones can be deleted by clicking x. To edit a difference, double-click it. The following popup will open:

Diff Yitra

First/Second Argument: What to compare. Normally, a name of a parameter is written here, surrounded by $ signs. You can enter characters and multiple parameter names.
Check For: Must be one of:
- newA - Only valid when A and B are lists or tables. Alerts on new rows* in A that do not exist in B.
- newB - Only valid when A and B are lists or tables. Alerts on new rows* in B that do not exist in A.
- change - If A and B are scalars (strings or numbers), alerts when they are different. If they are lists or tables their length should be the same, and Kamajii will alert when a row in A does not match the row of the same index in B.
- AgtB - Only valid when A and B are numbers. Alerts when A > B.
- BgtA - Only valid when A and B are numbers. Alerts when B > A.
- AeqB - Only valid when A and B are scalars (strings or numbers). Alerts when A and B are equal.
Type of action: What to do when a difference is found. Must be one of:
- notify - send a notification to the user and go on.
- error - send an error and quit the site (or go to the error page). The error is always sent to the standard error. In case the "Send alert?" checkbox in the Error Handling tab is checked, the error is sent to the user as well.
Urgency of Notification: The urgency rank can be used to control the order in which the messages are sent. More urgent messages will be sent before less urgent messages. 1 is the highest urgency rank, 2 is a little less urgent, etc.
Notification String: String that will be sent to the user's output destinations to notify that a difference was found. The string can contain parameters (surrounded by $ signs). It can also contain the values of the differring row in the data: $An$ is the value of the n'th column of this row in A, and $Bn$ is the value of the n'th column of this row in B. Note that the column index starts with 0 (i.e. the first column is marked as $A0$ or $B0$).

After you finish defining the site, save it from the File menu. Your users can then be notified on these pages.
Also, please consider sharing the site configurations with other people. The site configuration files can be found in the sites directory under your installation of Kamajii. If you email them to the Kamajii team, we will be glad to post them on the Kamajii site.

Note that the term "row" mentioned in the Differences section, has two meanings:

1) When the checked item is a list, "row" refers to one of the list's elements. Such a "row" contains only one column - the scalar value itself.

2) When the checked item is a table, "row" is a regular row in a table. Such a "row" can contain any number of columns.