Obtaining + Using WEKA

Obtaining Your Own Copy of Weka

You can obtain WEKA by visiting the WEKA Project Webpage and clicking on the appropriate link for your operating system.

Using Weka

These instructions describe how to apply different learning algorithms to the newsgroups data set. This assumes you have renderede the data into ARFF format using weka.py.

When you start up Weka, you will first see the WEKA GUI Chooser. You should click on the Explorer button. This opens a large panel with several tabs, and the Preprocess tab will already be selected.

Click on Open file..., navigate to the the newsgroups folder, and then select the diverse_train.arff file and click ok . The Current relation part of Explorer window should now show Relation as diverse with 50 instances and 101 attributes. All the attributes are listed on the left-hand side of the window. Now highlight Attribute 101 (Class), after which the table and bar plot on the right-hand side of the window will show 25 examples in class 1 and 25 in class 2.

Now click on the Classify tab of the Explorer window to select the learning algorithm to apply. Go to the Classifier panel (near the top) which initially shows ZeroR beside the Choose button. (ZeroR is a very simple rule-learning algorithm, which we do not want.) Clicking on Choose lets you choose a different algorithm.

Click on Choose, and you will see a hierarchical display with top level weka, second level classifiers, and with a third level containing seven general kinds of classifiers: bayes, functions, lazy, meta, misc, trees, and rules. You will be experimenting with the classifiers you think will work best by choosing from this list. To choose Naive Bayes, for example, click on the bayes indicator and then select NaiveBayes. You can examine the option for parameter settings by clikcing on the name of the algorithm in the window, which brings up another window with some of the options. Clicking on More in this window will bring up a description of the parameter options.

Now that you have chosen an algorithm, you need to examine the Test options panel. First load in the test data. Click on the radio button Supplied test set. Then click on the Set... button. A small Test Instances pop-up window should appear. Click on Open file..., navigate to the newsgroups folder, and then select the diverse_test.arff file and click ok. The Test Instances window should now show the relation diverse with 792 instances and 101 attributes. You may close this window at this point.

Down below there is a button which should read (Nom) newsgroup_class. This tells the algorithm the attribute to classify on.

Now you are ready to run the algorithm. Click on the Start button, and the Classifier Output window will show the output from the classifier. The output consists of several sections, most of which you don't need:

Run Information: Details of the data set and method of testing
Classifier model: The learned model, including statistics for each feature.
Evaluation on test set: This gives various statistics. The key item is the second one: Incorrectly Classified Instances will be expressed as a count and a percentage; this is the test error for the classifier. You should report the percentages in your answer. One other item of interest comes at the very end: the Confusion Matrix. This shows how many false positive and false negative errors were made.

After learning a classifier, you need to obtain the labels for the elements in the test set to hand in with your assignment. This is done as follows:

Choose the test set as above (Click the Set... button, etc.)
Click the More options... button
Check the Output predictions box
Click Start

This outputs the same as before, but also includes a bunch of predictions with the following form:

=== Predictions on test set ===

 inst#, actual, predicted, error, probability distribution
 1 1:rec.moto 1:rec.moto *1 0 
 2 1:rec.moto 1:rec.moto *1 0 
 3 1:rec.moto 1:rec.moto *1 0 
 4 1:rec.moto 1:rec.moto *1 0 
 5 1:rec.moto 1:rec.moto *1 0 
 6 1:rec.moto 1:rec.moto *1 0 
 7 1:rec.moto 1:rec.moto *1 0 
 8 1:rec.moto 1:rec.moto *1 0 
 9 1:rec.moto 1:rec.moto *1 0 
 10 1:rec.moto 1:rec.moto *1 0 
 11 1:rec.moto 1:rec.moto *1 0 
 12 1:rec.moto 1:rec.moto *1 0 
 13 1:rec.moto 1:rec.moto *1 0 
 ...

The + indicates an error between actual classification and predicted classification. The * flags the label with the highest probability --- ie, the value that would be returned here. After you have done this, you can either right-click (or middle click) on the Result list to save these labels to a text file, which will provide the results for handing in. Alternatively, just copy and paste the window contents to a file.