Skip to content
Snippets Groups Projects
Commit c1b1774e authored by ashivani's avatar ashivani
Browse files

updated baseline

parent 78e015f8
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
baselines/.ipynb_checkpoints/MNIST_baseline-checkpoint.ipynb
+ 12
− 37
View file @ c1b1774e
%% Cell type:markdown id: tags:
# Baseline for MNIST Educational Challenge on AIcrowd
#### Author : Ayush Shivani
%% Cell type:markdown id: tags:
## To open this notebook on Google Computing platform Colab, click below!
%% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ayushshivani/aicrowd_educational_baselines/blob/master/MNIST_baseline.ipynb)
%% Cell type:markdown id: tags:
## Download Necessary Packages
%% Cell type:code id: tags:
``` python
import sys
!{sys.executable} -m pip install numpy
!{sys.executable} -m pip install pandas
!{sys.executable} -m pip install scikit-learn
```
%% Cell type:markdown id: tags:
## Download data
The first step is to download out train test data. We will be training a classifier on the train data and make predictions on test data. We submit our predictions
%% Cell type:code id: tags:
``` python
!wget https://s3.eu-central-1.wasabisys.com/aicrowd-public-datasets/aicrowd_educational_mnist/data/public/test.zip
!wget https://s3.eu-central-1.wasabisys.com/aicrowd-public-datasets/aicrowd_educational_mnist/data/public/train.zip
!unzip train.zip
!unzip test.zip
```
%% Cell type:markdown id: tags:
## Import packages
%% Cell type:code id: tags:
``` python
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.metrics import f1_score,precision_score,recall_score,accuracy_score
```
%% Cell type:markdown id: tags:
## Load Data
We use pandas library to load our data. Pandas loads them into dataframes which helps us analyze our data easily. Learn more about it [here](https://www.tutorialspoint.com/python_data_science/python_pandas.htm)
%% Cell type:code id: tags:
``` python
train_data_path = "train.csv" #path where data is stored
```
%% Cell type:code id: tags:
``` python
train_data = pd.read_csv(train_data_path,header=None) #load data in dataframe using pandas
```
%% Cell type:markdown id: tags:
## Visualise the Dataset
%% Cell type:code id: tags:
``` python
train_data.head()
```
%% Cell type:markdown id: tags:
You can see the columns goes from 0 to 784, where columns from 1 to 784 denotes pixel values each between 0-255 and the first column i.e. 0th is the digit it represents between 0-9.
%% Cell type:markdown id: tags:
## Split Data into Train and Validation
Now we want to see how well our classifier is performing, but we dont have the test data labels with us to check. What do we do ? So we split our dataset into train and validation. The idea is that we test our classifier on validation set in order to get an idea of how well our classifier works. This way we can also ensure that we dont [overfit](https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/) on the train dataset. There are many ways to do validation like [k-fold](https://machinelearningmastery.com/k-fold-cross-validation/),[leave one out](https://en.wikipedia.org/wiki/Cross-validation_(statistics), etc## Split the data in train/test
%% Cell type:code id: tags:
``` python
X_train, X_val= train_test_split(train_data, test_size=0.2, random_state=42)
```
%% Output
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-1-a7c485c3cf8f> in <module>
----> 1 X_train, X_val= train_test_split(train_data, test_size=0.2, random_state=42)
NameError: name 'train_test_split' is not defined
%% Cell type:markdown id: tags:
Here we have selected the size of the testing data to be 20% of the total data. You can change it and see what effect it has on the accuracies. To learn more about the train_test_split function [click here](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html).
%% Cell type:markdown id: tags:
Now, since we have our data splitted into train and validation sets, we need to get the label separated from the data.
%% Cell type:code id: tags:
``` python
X_train,y_train = X_train.iloc[:,1:],X_train.iloc[:,0]
X_val,y_val = X_val.iloc[:,1:],X_val.iloc[:,0]
```
%% Cell type:markdown id: tags:
## Define the Classifier
We have fixed our data and now we train a classifier. The classifier will learn the function by looking at the inputs and corresponding outputs. There are a ton of classifiers to choose from some being [Logistic Regression](https://towardsdatascience.com/logistic-regression-detailed-overview-46c4da4303bc), [SVM](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Random Forests](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Decision Trees](https://towardsdatascience.com/decision-trees-in-machine-learning-641b9c4e8052), etc.
Tip: A good model doesnt depend solely on the classifier but on the features(columns) you choose. So make sure to play with your data and keep only whats important.
%% Cell type:code id: tags:
``` python
classifier = LogisticRegression(solver = 'lbfgs',multi_class='auto',max_iter=10)
# from sklearn.svm import SVC
# classifier = SVC(gamma='auto')
# from sklearn import tree
# classifier = tree.DecisionTreeClassifier()
```
%% Cell type:markdown id: tags:
We have used [Logistic Regression](https://en.wikipedia.org/wiki/Logistic_regression) as a classifier here and set few of the parameteres. But one can set more parameters and increase the performance. To see the list of parameters visit [here](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html).
Also given are SVM and Decision Tree examples. Check out SVM's parameters [here](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html) and Decision Tree's [here](https://scikit-learn.org/stable/modules/tree.html)
%% Cell type:markdown id: tags:
We can also use other classifiers. To read more about sklean classifiers visit [here](https://scikit-learn.org/stable/supervised_learning.html). Try and use other classifiers to see how the performance of your model changes.
%% Cell type:markdown id: tags:
## Train the classifier
%% Cell type:code id: tags:
``` python
classifier.fit(X_train, y_train)
```
%% Cell type:markdown id: tags:
Got a warning! Dont worry, its just beacuse the number of iteration is very less(defined in the classifier in the above cell).Increase the number of iterations and see if the warning vanishes.Do remember increasing iterations also increases the running time.( Hint: max_iter=500)
%% Cell type:markdown id: tags:
## Predict on Validation
Now we predict our trained classifier on the validation set and evaluate our model
%% Cell type:code id: tags:
``` python
y_pred = classifier.predict(X_test)
y_pred = classifier.predict(X_val)
```
%% Cell type:markdown id: tags:
## Evaluate the Performance
We use the same metrics as that will be used for the test set.
[F1 score](https://en.wikipedia.org/wiki/F1_score) and [Log Loss](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html) are the metrics for this challenge
%% Cell type:code id: tags:
``` python
precision = precision_score(y_val,y_pred,average='micro')
recall = recall_score(y_val,y_pred,average='micro')
accuracy = accuracy_score(y_val,y_pred)
f1 = f1_score(y_val,y_pred,average='macro')
```
%% Cell type:code id: tags:
``` python
print("Accuracy of the model is :" ,accuracy)
print("Recall of the model is :" ,recall)
print("Precision of the model is :" ,precision)
print("F1 score of the model is :" ,f1)
```
%% Output
Accuracy of the model is : 0.89356210142727
Recall of the model is : 0.89356210142727
Precision of the model is : 0.89356210142727
F1 score of the model is : 0.6058161925441079
%% Cell type:markdown id: tags:
Here are some of the images predicted correctly by your model.Cheers!
%% Cell type:code id: tags:
``` python
import matplotlib.pyplot as plt
def correctly_predicted():
correct_pred = np.where(y_pred == y_test)
correct_pred = np.where(y_pred == y_val)
fig = plt.figure()
for i in range(1,10):
img = X_test.iloc[i,:]
img = X_val.iloc[i,:]
img = np.array(img).reshape(28,28)
fig.add_subplot(3,3,i)
plt.imshow(img,cmap='gray')
correctly_predicted()
```
%% Cell type:markdown id: tags:
# Prediction on Evaluation Set
%% Cell type:markdown id: tags:
## Load Test Set
Load the test data now
%% Cell type:code id: tags:
``` python
final_test_path = "test.csv"
final_test = pd.read_csv(final_test_path)
final_test = pd.read_csv(final_test_path,header=None)
```
%% Cell type:markdown id: tags:
## Predict Test Set
Time for the moment of truth! Predict on test set and time to make the submission.
%% Cell type:code id: tags:
``` python
submission = classifier.predict(final_test)
```
%% Cell type:markdown id: tags:
## Save the prediction to csv
%% Cell type:code id: tags:
``` python
submission = pd.DataFrame(submission)
submission.to_csv('/tmp/submission.csv',header=['label'],index=False)
```
%% Cell type:markdown id: tags:
Note: Do take a look at the submission format.The submission file should contain a header.For eg here it is "label".
%% Cell type:markdown id: tags:
## To download the generated csv in colab run the below command
%% Cell type:code id: tags:
``` python
from google.colab import files
files.download('/tmp/submission.csv')
```
%% Cell type:markdown id: tags:
### Go to [platform](https://www.aicrowd.com/challenges/mnist-recognise-handwritten-digits/). Participate in the challenge and submit the submission.csv generated.
......
baselines/MNIST_baseline.ipynb
+ 12
− 37
View file @ c1b1774e
%% Cell type:markdown id: tags:
# Baseline for MNIST Educational Challenge on AIcrowd
#### Author : Ayush Shivani
%% Cell type:markdown id: tags:
## To open this notebook on Google Computing platform Colab, click below!
%% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ayushshivani/aicrowd_educational_baselines/blob/master/MNIST_baseline.ipynb)
%% Cell type:markdown id: tags:
## Download Necessary Packages
%% Cell type:code id: tags:
``` python
import sys
!{sys.executable} -m pip install numpy
!{sys.executable} -m pip install pandas
!{sys.executable} -m pip install scikit-learn
```
%% Cell type:markdown id: tags:
## Download data
The first step is to download out train test data. We will be training a classifier on the train data and make predictions on test data. We submit our predictions
%% Cell type:code id: tags:
``` python
!wget https://s3.eu-central-1.wasabisys.com/aicrowd-public-datasets/aicrowd_educational_mnist/data/public/test.zip
!wget https://s3.eu-central-1.wasabisys.com/aicrowd-public-datasets/aicrowd_educational_mnist/data/public/train.zip
!unzip train.zip
!unzip test.zip
```
%% Cell type:markdown id: tags:
## Import packages
%% Cell type:code id: tags:
``` python
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.metrics import f1_score,precision_score,recall_score,accuracy_score
```
%% Cell type:markdown id: tags:
## Load Data
We use pandas library to load our data. Pandas loads them into dataframes which helps us analyze our data easily. Learn more about it [here](https://www.tutorialspoint.com/python_data_science/python_pandas.htm)
%% Cell type:code id: tags:
``` python
train_data_path = "train.csv" #path where data is stored
```
%% Cell type:code id: tags:
``` python
train_data = pd.read_csv(train_data_path,header=None) #load data in dataframe using pandas
```
%% Cell type:markdown id: tags:
## Visualise the Dataset
%% Cell type:code id: tags:
``` python
train_data.head()
```
%% Cell type:markdown id: tags:
You can see the columns goes from 0 to 784, where columns from 1 to 784 denotes pixel values each between 0-255 and the first column i.e. 0th is the digit it represents between 0-9.
%% Cell type:markdown id: tags:
## Split Data into Train and Validation
Now we want to see how well our classifier is performing, but we dont have the test data labels with us to check. What do we do ? So we split our dataset into train and validation. The idea is that we test our classifier on validation set in order to get an idea of how well our classifier works. This way we can also ensure that we dont [overfit](https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/) on the train dataset. There are many ways to do validation like [k-fold](https://machinelearningmastery.com/k-fold-cross-validation/),[leave one out](https://en.wikipedia.org/wiki/Cross-validation_(statistics), etc## Split the data in train/test
%% Cell type:code id: tags:
``` python
X_train, X_val= train_test_split(train_data, test_size=0.2, random_state=42)
```
%% Output
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-1-a7c485c3cf8f> in <module>
----> 1 X_train, X_val= train_test_split(train_data, test_size=0.2, random_state=42)
NameError: name 'train_test_split' is not defined
%% Cell type:markdown id: tags:
Here we have selected the size of the testing data to be 20% of the total data. You can change it and see what effect it has on the accuracies. To learn more about the train_test_split function [click here](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html).
%% Cell type:markdown id: tags:
Now, since we have our data splitted into train and validation sets, we need to get the label separated from the data.
%% Cell type:code id: tags:
``` python
X_train,y_train = X_train.iloc[:,1:],X_train.iloc[:,0]
X_val,y_val = X_val.iloc[:,1:],X_val.iloc[:,0]
```
%% Cell type:markdown id: tags:
## Define the Classifier
We have fixed our data and now we train a classifier. The classifier will learn the function by looking at the inputs and corresponding outputs. There are a ton of classifiers to choose from some being [Logistic Regression](https://towardsdatascience.com/logistic-regression-detailed-overview-46c4da4303bc), [SVM](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Random Forests](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Decision Trees](https://towardsdatascience.com/decision-trees-in-machine-learning-641b9c4e8052), etc.
Tip: A good model doesnt depend solely on the classifier but on the features(columns) you choose. So make sure to play with your data and keep only whats important.
%% Cell type:code id: tags:
``` python
classifier = LogisticRegression(solver = 'lbfgs',multi_class='auto',max_iter=10)
# from sklearn.svm import SVC
# classifier = SVC(gamma='auto')
# from sklearn import tree
# classifier = tree.DecisionTreeClassifier()
```
%% Cell type:markdown id: tags:
We have used [Logistic Regression](https://en.wikipedia.org/wiki/Logistic_regression) as a classifier here and set few of the parameteres. But one can set more parameters and increase the performance. To see the list of parameters visit [here](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html).
Also given are SVM and Decision Tree examples. Check out SVM's parameters [here](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html) and Decision Tree's [here](https://scikit-learn.org/stable/modules/tree.html)
%% Cell type:markdown id: tags:
We can also use other classifiers. To read more about sklean classifiers visit [here](https://scikit-learn.org/stable/supervised_learning.html). Try and use other classifiers to see how the performance of your model changes.
%% Cell type:markdown id: tags:
## Train the classifier
%% Cell type:code id: tags:
``` python
classifier.fit(X_train, y_train)
```
%% Cell type:markdown id: tags:
Got a warning! Dont worry, its just beacuse the number of iteration is very less(defined in the classifier in the above cell).Increase the number of iterations and see if the warning vanishes.Do remember increasing iterations also increases the running time.( Hint: max_iter=500)
%% Cell type:markdown id: tags:
## Predict on Validation
Now we predict our trained classifier on the validation set and evaluate our model
%% Cell type:code id: tags:
``` python
y_pred = classifier.predict(X_test)
y_pred = classifier.predict(X_val)
```
%% Cell type:markdown id: tags:
## Evaluate the Performance
We use the same metrics as that will be used for the test set.
[F1 score](https://en.wikipedia.org/wiki/F1_score) and [Log Loss](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html) are the metrics for this challenge
%% Cell type:code id: tags:
``` python
precision = precision_score(y_val,y_pred,average='micro')
recall = recall_score(y_val,y_pred,average='micro')
accuracy = accuracy_score(y_val,y_pred)
f1 = f1_score(y_val,y_pred,average='macro')
```
%% Cell type:code id: tags:
``` python
print("Accuracy of the model is :" ,accuracy)
print("Recall of the model is :" ,recall)
print("Precision of the model is :" ,precision)
print("F1 score of the model is :" ,f1)
```
%% Output
Accuracy of the model is : 0.89356210142727
Recall of the model is : 0.89356210142727
Precision of the model is : 0.89356210142727
F1 score of the model is : 0.6058161925441079
%% Cell type:markdown id: tags:
Here are some of the images predicted correctly by your model.Cheers!
%% Cell type:code id: tags:
``` python
import matplotlib.pyplot as plt
def correctly_predicted():
correct_pred = np.where(y_pred == y_test)
correct_pred = np.where(y_pred == y_val)
fig = plt.figure()
for i in range(1,10):
img = X_test.iloc[i,:]
img = X_val.iloc[i,:]
img = np.array(img).reshape(28,28)
fig.add_subplot(3,3,i)
plt.imshow(img,cmap='gray')
correctly_predicted()
```
%% Cell type:markdown id: tags:
# Prediction on Evaluation Set
%% Cell type:markdown id: tags:
## Load Test Set
Load the test data now
%% Cell type:code id: tags:
``` python
final_test_path = "test.csv"
final_test = pd.read_csv(final_test_path)
final_test = pd.read_csv(final_test_path,header=None)
```
%% Cell type:markdown id: tags:
## Predict Test Set
Time for the moment of truth! Predict on test set and time to make the submission.
%% Cell type:code id: tags:
``` python
submission = classifier.predict(final_test)
```
%% Cell type:markdown id: tags:
## Save the prediction to csv
%% Cell type:code id: tags:
``` python
submission = pd.DataFrame(submission)
submission.to_csv('/tmp/submission.csv',header=['label'],index=False)
```
%% Cell type:markdown id: tags:
Note: Do take a look at the submission format.The submission file should contain a header.For eg here it is "label".
%% Cell type:markdown id: tags:
## To download the generated csv in colab run the below command
%% Cell type:code id: tags:
``` python
from google.colab import files
files.download('/tmp/submission.csv')
```
%% Cell type:markdown id: tags:
### Go to [platform](https://www.aicrowd.com/challenges/mnist-recognise-handwritten-digits/). Participate in the challenge and submit the submission.csv generated.
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment