Py: Neural Net Classification

Py: Neural Net Classification#

This notebook was originally created by Hugh Miller for the Data Analytics Applications subject, as Exercise 5.18 - Spam detection with a neural network in the DAA M05 Classification and neural networks module.

Data Analytics Applications is a Fellowship Applications (Module 3) subject with the Actuaries Institute that aims to teach students how to apply a range of data analytics skills, such as neural networks, natural language processing, unsupervised learning and optimisation techniques, together with their professional judgement, to solve a variety of complex and challenging business problems. The business problems used as examples in this subject are drawn from a wide range of industries.

Find out more about the course here.

Purpose:#

This notebook fits a neural network on a spam dataset to identify spam emails based on features about the email.

References:#

The spam dataset is sourced from the University of California, Irvine Machine Learning Repository: Hopkins, M., Reeber, E., Forman, G., and Suermondt, J. (1999). Spambase Data Set [Dataset]. https://archive.ics.uci.edu/ml/datasets/Spambase.

This dataset contains the following:

4,601 observations, each representing an email originally collected from a Hewlett-Packard email server, of which 1,813 (39%) were identified as spam;
57 continuous features:
- 48 features of type ‘word_freq_WORD’ that represent the percentage (0 to 100) of words in the email that match ‘WORD’;
- 6 features of type ‘char_freq_CHAR’ that represent the percentage (0 to 100) of characters in the email that match ‘CHAR’;
- 1 feature, ‘capital_run_length_average’, that is the average length of uninterrupted sequences of capital letters in the email;
- 1 feature, ‘capital_run_length_longest’, that is the length of the longest uninterrupted sequence of capital letters in the email; and
- 1 feature, ‘capital_run_length_total’, that is the total number of capital letters in the email; and
a binary response variable that takes on a value 0 if the email is not spam and 1 if the email is spam.

Packages#

This section imports the packages that will be required for this exercise/case study.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# Used to build the neural network and evaluate it.
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.preprocessing import OneHotEncoder
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense

Data#

This section:

imports the data that will be used in the modelling; and
explores the data.

Import data#

# Create a list of headings for the data.
namearray = [
'word_freq_make',
'word_freq_address',
'word_freq_all',
'word_freq_3d',
'word_freq_our',
'word_freq_over',
'word_freq_remove',
'word_freq_internet',
'word_freq_order',
'word_freq_mail',
'word_freq_receive',
'word_freq_will',
'word_freq_people',
'word_freq_report',
'word_freq_addresses',
'word_freq_free',
'word_freq_business',
'word_freq_email',
'word_freq_you',
'word_freq_credit',
'word_freq_your',
'word_freq_font',
'word_freq_000',
'word_freq_money',
'word_freq_hp',
'word_freq_hpl',
'word_freq_george',
'word_freq_650',
'word_freq_lab',
'word_freq_labs',
'word_freq_telnet',
'word_freq_857',
'word_freq_data',
'word_freq_415',
'word_freq_85',
'word_freq_technology',
'word_freq_1999',
'word_freq_parts',
'word_freq_pm',
'word_freq_direct',
'word_freq_cs',
'word_freq_meeting',
'word_freq_original',
'word_freq_project',
'word_freq_re',
'word_freq_edu',
'word_freq_table',
'word_freq_conference',
'char_freq_;',
'char_freq_(',
'char_freq_[',
'char_freq_!',
'char_freq_$',
'char_freq_#',
'capital_run_length_average',
'capital_run_length_longest',
'capital_run_length_total',
'Spam_fl' ]

# Read in the data from the Stanford website.
spam = pd.read_csv("http://www.web.stanford.edu/~hastie/ElemStatLearn/datasets/spam.data", delim_whitespace=True, 
                   header=None,
                   names=namearray
                 )

Explore data (EDA)#

# Check the dimensions of the data.
print(spam.info())

# Print the first 10 observations from the data.
print(spam.head(10))

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 4601 entries, 0 to 4600
Data columns (total 58 columns):
 #   Column                      Non-Null Count  Dtype  
---  ------                      --------------  -----  
 word_freq_make              4601 non-null   float64
 word_freq_address           4601 non-null   float64
 word_freq_all               4601 non-null   float64
 word_freq_3d                4601 non-null   float64
 word_freq_our               4601 non-null   float64
 word_freq_over              4601 non-null   float64
 word_freq_remove            4601 non-null   float64
 word_freq_internet          4601 non-null   float64
 word_freq_order             4601 non-null   float64
 word_freq_mail              4601 non-null   float64
word_freq_receive           4601 non-null   float64
word_freq_will              4601 non-null   float64
word_freq_people            4601 non-null   float64
word_freq_report            4601 non-null   float64
word_freq_addresses         4601 non-null   float64
word_freq_free              4601 non-null   float64
word_freq_business          4601 non-null   float64
word_freq_email             4601 non-null   float64
word_freq_you               4601 non-null   float64
word_freq_credit            4601 non-null   float64
word_freq_your              4601 non-null   float64
word_freq_font              4601 non-null   float64
word_freq_000               4601 non-null   float64
word_freq_money             4601 non-null   float64
word_freq_hp                4601 non-null   float64
word_freq_hpl               4601 non-null   float64
word_freq_george            4601 non-null   float64
word_freq_650               4601 non-null   float64
word_freq_lab               4601 non-null   float64
word_freq_labs              4601 non-null   float64
word_freq_telnet            4601 non-null   float64
word_freq_857               4601 non-null   float64
word_freq_data              4601 non-null   float64
word_freq_415               4601 non-null   float64
word_freq_85                4601 non-null   float64
word_freq_technology        4601 non-null   float64
word_freq_1999              4601 non-null   float64
word_freq_parts             4601 non-null   float64
word_freq_pm                4601 non-null   float64
word_freq_direct            4601 non-null   float64
word_freq_cs                4601 non-null   float64
word_freq_meeting           4601 non-null   float64
word_freq_original          4601 non-null   float64
word_freq_project           4601 non-null   float64
word_freq_re                4601 non-null   float64
word_freq_edu               4601 non-null   float64
word_freq_table             4601 non-null   float64
word_freq_conference        4601 non-null   float64
char_freq_;                 4601 non-null   float64
char_freq_(                 4601 non-null   float64
char_freq_[                 4601 non-null   float64
char_freq_!                 4601 non-null   float64
char_freq_$                 4601 non-null   float64
char_freq_#                 4601 non-null   float64
capital_run_length_average  4601 non-null   float64
capital_run_length_longest  4601 non-null   int64  
capital_run_length_total    4601 non-null   int64  
Spam_fl                     4601 non-null   int64  
dtypes: float64(55), int64(3)
memory usage: 2.0 MB
None
   word_freq_make  word_freq_address  word_freq_all  word_freq_3d  \
          0.00               0.64           0.64           0.0   
          0.21               0.28           0.50           0.0   
          0.06               0.00           0.71           0.0   
          0.00               0.00           0.00           0.0   
          0.00               0.00           0.00           0.0   
          0.00               0.00           0.00           0.0   
          0.00               0.00           0.00           0.0   
          0.00               0.00           0.00           0.0   
          0.15               0.00           0.46           0.0   
          0.06               0.12           0.77           0.0   

   word_freq_our  word_freq_over  word_freq_remove  word_freq_internet  \
         0.32            0.00              0.00                0.00   
         0.14            0.28              0.21                0.07   
         1.23            0.19              0.19                0.12   
         0.63            0.00              0.31                0.63   
         0.63            0.00              0.31                0.63   
         1.85            0.00              0.00                1.85   
         1.92            0.00              0.00                0.00   
         1.88            0.00              0.00                1.88   
         0.61            0.00              0.30                0.00   
         0.19            0.32              0.38                0.00   

   word_freq_order  word_freq_mail  ...  char_freq_;  char_freq_(  \
           0.00            0.00  ...         0.00        0.000   
           0.00            0.94  ...         0.00        0.132   
           0.64            0.25  ...         0.01        0.143   
           0.31            0.63  ...         0.00        0.137   
           0.31            0.63  ...         0.00        0.135   
           0.00            0.00  ...         0.00        0.223   
           0.00            0.64  ...         0.00        0.054   
           0.00            0.00  ...         0.00        0.206   
           0.92            0.76  ...         0.00        0.271   
           0.06            0.00  ...         0.04        0.030   

   char_freq_[  char_freq_!  char_freq_$  char_freq_#  \
        0.0        0.778        0.000        0.000   
        0.0        0.372        0.180        0.048   
        0.0        0.276        0.184        0.010   
        0.0        0.137        0.000        0.000   
        0.0        0.135        0.000        0.000   
        0.0        0.000        0.000        0.000   
        0.0        0.164        0.054        0.000   
        0.0        0.000        0.000        0.000   
        0.0        0.181        0.203        0.022   
        0.0        0.244        0.081        0.000   

   capital_run_length_average  capital_run_length_longest  \
                     3.756                          61   
                     5.114                         101   
                     9.821                         485   
                     3.537                          40   
                     3.537                          40   
                     3.000                          15   
                     1.671                           4   
                     2.450                          11   
                     9.744                         445   
                     1.729                          43   

   capital_run_length_total  Spam_fl  
                     278        1  
                    1028        1  
                    2259        1  
                     191        1  
                     191        1  
                      54        1  
                     112        1  
                      49        1  
                    1257        1  
                     749        1  

[10 rows x 58 columns]

Modelling#