Skip to content
Snippets Groups Projects
Commit f17971c9 authored by ashivani's avatar ashivani
Browse files

changed baseline

parent 03866fb1
No related branches found
No related tags found
No related merge requests found
Pipeline #4125 failed
Hide whitespace changes
Inline Side-by-side
baselines/.ipynb_checkpoints/MNIST_baseline-checkpoint.ipynb
+ 4
− 24
View file @ f17971c9
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
![AIcrowd-Logo](https://raw.githubusercontent.com/AIcrowd/AIcrowd/master/app/assets/images/misc/aicrowd-horizontal.png) ![AIcrowd-Logo](https://raw.githubusercontent.com/AIcrowd/AIcrowd/master/app/assets/images/misc/aicrowd-horizontal.png)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Getting Started Code for [MNIST Educational Challenge](https://www.aicrowd.com/challenges/mnist) # Getting Started Code for [MNIST Educational Challenge](https://www.aicrowd.com/challenges/mnist)
#### Author : Ayush Shivani #### Author : Ayush Shivani
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## To open this notebook on Google Computing platform Colab, click below! ## To open this notebook on Google Computing platform Colab, click below!
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ayushshivani/aicrowd_educational_baselines/blob/master/MNIST_baseline.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ayushshivani/aicrowd_educational_baselines/blob/master/MNIST_baseline.ipynb)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Download Necessary Packages 📚 ## Download Necessary Packages 📚
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import sys import sys
!pip install numpy !pip install numpy
!pip install pandas !pip install pandas
!pip install scikit-learn !pip install scikit-learn
!pip install matplotlib !pip install matplotlib
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Download Data ## Download Data
The first step is to download out train test data. We will be training a model on the train data and make predictions on test data. We submit our predictions. The first step is to download out train test data. We will be training a model on the train data and make predictions on test data. We submit our predictions.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
#Donwload the datasets #Donwload the datasets
!rm -rf data !rm -rf data
!mkdir data !mkdir data
!wget https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/mnist/v0.1/test.zip !wget https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/mnist/v0.1/test.zip
!wget https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/mnist/v0.1/train.zip !wget https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/mnist/v0.1/train.zip
!unzip test.zip !unzip test.zip
!unzip test.zip
!mv test.csv data/test.csv !mv test.csv data/test.csv
!mv train.csv data/train.csv !mv train.csv data/train.csv
!rm test.zip train.zip !rm test.zip train.zip
``` ```
%% Output
--2020-05-17 00:00:14-- https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/mnist/v0.1/test.zip
Resolving s3.eu-central-1.wasabisys.com (s3.eu-central-1.wasabisys.com)... 130.117.252.16, 130.117.252.12, 130.117.252.13, ...
Connecting to s3.eu-central-1.wasabisys.com (s3.eu-central-1.wasabisys.com)|130.117.252.16|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2202644 (2.1M) [application/zip]
Saving to: ‘test.zip’
test.zip 100%[===================>] 2.10M 64.3KB/s in 29s
2020-05-17 00:00:44 (74.3 KB/s) - ‘test.zip’ saved [2202644/2202644]
Archive: test.zip
inflating: test.csv
rm: cannot remove 'train.zip': No such file or directory
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Import packages ## Import packages
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import pandas as pd import pandas as pd
import numpy as np import numpy as np
from sklearn.model_selection import train_test_split from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC from sklearn.svm import SVC
from sklearn.metrics import f1_score,precision_score,recall_score,accuracy_score from sklearn.metrics import f1_score,precision_score,recall_score,accuracy_score
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Load Data ## Load Data
- We use pandas 🐼 library to load our data. - We use pandas 🐼 library to load our data.
- Pandas loads the data into dataframes and facilitates us to analyse the data. - Pandas loads the data into dataframes and facilitates us to analyse the data.
- Learn more about it [here](https://www.tutorialspoint.com/python_data_science/python_pandas.htm) 🤓 - Learn more about it [here](https://www.tutorialspoint.com/python_data_science/python_pandas.htm) 🤓
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
all_data_path = "data/train.csv" #path where data is stored all_data_path = "data/train.csv" #path where data is stored
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
all_data = pd.read_csv(all_data_path,header=None) #load data in dataframe using pandas all_data = pd.read_csv(all_data_path,header=None) #load data in dataframe using pandas
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Visualize the data 👀 ## Visualize the data 👀
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
all_data.head() all_data.head()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
You can see the columns goes from 0 to 784, where columns from 1 to 784 denotes pixel values each between 0-255 and the first column i.e. 0th is the digit it represents between 0-9. You can see the columns goes from 0 to 784, where columns from 1 to 784 denotes pixel values each between 0-255 and the first column i.e. 0th is the digit it represents between 0-9.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Split Data into Train and Validation 🔪 ## Split Data into Train and Validation 🔪
- The next step is to think of a way to test how well our model is performing. we cannot use the test data given as it does not contain the data labels for us to verify. - The next step is to think of a way to test how well our model is performing. we cannot use the test data given as it does not contain the data labels for us to verify.
- The workaround this is to split the given training data into training and validation. Typically validation sets give us an idea of how our model will perform on unforeseen data. it is like holding back a chunk of data while training our model and then using it to for the purpose of testing. it is a standard way to fine-tune hyperparameters in a model. - The workaround this is to split the given training data into training and validation. Typically validation sets give us an idea of how our model will perform on unforeseen data. it is like holding back a chunk of data while training our model and then using it to for the purpose of testing. it is a standard way to fine-tune hyperparameters in a model.
- There are multiple ways to split a dataset into validation and training sets. following are two popular ways to go about it, [k-fold](https://machinelearningmastery.com/k-fold-cross-validation/), [leave one out](https://en.wikipedia.org/wiki/Cross-validation_statistics). 🧐 - There are multiple ways to split a dataset into validation and training sets. following are two popular ways to go about it, [k-fold](https://machinelearningmastery.com/k-fold-cross-validation/), [leave one out](https://en.wikipedia.org/wiki/Cross-validation_statistics). 🧐
- Validation sets are also used to avoid your model from [overfitting](https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/) on the train dataset. - Validation sets are also used to avoid your model from [overfitting](https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/) on the train dataset.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
X_train, X_val= train_test_split(all_data, test_size=0.2, random_state=42) X_train, X_val= train_test_split(all_data, test_size=0.2, random_state=42)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
- We have decided to split the data with 20 % as validation and 80 % as training. - We have decided to split the data with 20 % as validation and 80 % as training.
- To learn more about the train_test_split function [click here](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html). 🧐 - To learn more about the train_test_split function [click here](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html). 🧐
- This is of course the simplest way to validate your model by simply taking a random chunk of the train set and setting it aside solely for the purpose of testing our train model on unseen data. as mentioned in the previous block, you can experiment 🔬 with and choose more sophisticated techniques and make your model better. - This is of course the simplest way to validate your model by simply taking a random chunk of the train set and setting it aside solely for the purpose of testing our train model on unseen data. as mentioned in the previous block, you can experiment 🔬 with and choose more sophisticated techniques and make your model better.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
- Now, since we have our data splitted into train and validation sets, we need to get the corresponding labels separated from the data. - Now, since we have our data splitted into train and validation sets, we need to get the corresponding labels separated from the data.
- with this step we are all set move to the next step with a prepared dataset. - with this step we are all set move to the next step with a prepared dataset.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
X_train,y_train = X_train.iloc[:,1:],X_train.iloc[:,0] X_train,y_train = X_train.iloc[:,1:],X_train.iloc[:,0]
X_val,y_val = X_val.iloc[:,1:],X_val.iloc[:,0] X_val,y_val = X_val.iloc[:,1:],X_val.iloc[:,0]
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# TRAINING PHASE 🏋️ # TRAINING PHASE 🏋️
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Define the Model ## Define the Model
- We have fixed our data and now we are ready to train our model. - We have fixed our data and now we are ready to train our model.
- There are a ton of classifiers to choose from some being [Logistic Regression](https://towardsdatascience.com/logistic-regression-detailed-overview-46c4da4303bc), [SVM](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Random Forests](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Decision Trees](https://towardsdatascience.com/decision-trees-in-machine-learning-641b9c4e8052), etc.🧐 - There are a ton of classifiers to choose from some being [Logistic Regression](https://towardsdatascience.com/logistic-regression-detailed-overview-46c4da4303bc), [SVM](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Random Forests](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Decision Trees](https://towardsdatascience.com/decision-trees-in-machine-learning-641b9c4e8052), etc.🧐
- Remember that there are no hard-laid rules here. you can mix and match classifiers, it is advisable to read up on the numerous techniques and choose the best fit for your solution , experimentation is the key. - Remember that there are no hard-laid rules here. you can mix and match classifiers, it is advisable to read up on the numerous techniques and choose the best fit for your solution , experimentation is the key.
- A good model does not depend solely on the classifier but also on the features you choose. So make sure to analyse and understand your data well and move forward with a clear view of the problem at hand. you can gain important insight from [here](https://towardsdatascience.com/the-5-feature-selection-algorithms-every-data-scientist-need-to-know-3a6b566efd2).🧐 - A good model does not depend solely on the classifier but also on the features you choose. So make sure to analyse and understand your data well and move forward with a clear view of the problem at hand. you can gain important insight from [here](https://towardsdatascience.com/the-5-feature-selection-algorithms-every-data-scientist-need-to-know-3a6b566efd2).🧐
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
classifier = LogisticRegression(solver = 'lbfgs',multi_class='auto',max_iter=10) classifier = LogisticRegression(solver = 'lbfgs',multi_class='auto',max_iter=10)
# from sklearn.svm import SVC # from sklearn.svm import SVC
# classifier = SVC(gamma='auto') # classifier = SVC(gamma='auto')
# from sklearn import tree # from sklearn import tree
# classifier = tree.DecisionTreeClassifier() # classifier = tree.DecisionTreeClassifier()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
- To start you off, We have used a basic [Support Vector Machines](https://scikit-learn.org/stable/modules/svm.html#classification) classifier here. - To start you off, We have used a basic [Support Vector Machines](https://scikit-learn.org/stable/modules/svm.html#classification) classifier here.
- But you can tune parameters and increase the performance. To see the list of parameters visit [here](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html). - But you can tune parameters and increase the performance. To see the list of parameters visit [here](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html).
- Do keep in mind there exist sophisticated techniques for everything, the key as quoted earlier is to search them and experiment to fit your implementation. - Do keep in mind there exist sophisticated techniques for everything, the key as quoted earlier is to search them and experiment to fit your implementation.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
To read more about other sklearn classifiers visit [here 🧐](https://scikit-learn.org/stable/supervised_learning.html). Try and use other classifiers to see how the performance of your model changes. Try using [Logistic Regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) or [MLP](http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html) and compare how the performance changes. To read more about other sklearn classifiers visit [here 🧐](https://scikit-learn.org/stable/supervised_learning.html). Try and use other classifiers to see how the performance of your model changes. Try using [Logistic Regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) or [MLP](http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html) and compare how the performance changes.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Train the Model ## Train the Model
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
classifier.fit(X_train, y_train) classifier.fit(X_train, y_train)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Got a warning! Dont worry, its just beacuse the number of iteration is very less(defined in the classifier in the above cell).Increase the number of iterations and see if the warning vanishes.Do remember increasing iterations also increases the running time.( Hint: max_iter=500) Got a warning! Dont worry, its just beacuse the number of iteration is very less(defined in the classifier in the above cell).Increase the number of iterations and see if the warning vanishes.Do remember increasing iterations also increases the running time.( Hint: max_iter=500)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Validation Phase 🤔 # Validation Phase 🤔
Wonder how well your model learned! Lets check it. Wonder how well your model learned! Lets check it.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Predict on Validation ## Predict on Validation
Now we predict using our trained model on the validation set we created and evaluate our model on unforeseen data. Now we predict using our trained model on the validation set we created and evaluate our model on unforeseen data.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
y_pred = classifier.predict(X_val) y_pred = classifier.predict(X_val)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Evaluate the Performance ## Evaluate the Performance
- We have used basic metrics to quantify the performance of our model. - We have used basic metrics to quantify the performance of our model.
- This is a crucial step, you should reason out the metrics and take hints to improve aspects of your model. - This is a crucial step, you should reason out the metrics and take hints to improve aspects of your model.
- Do read up on the meaning and use of different metrics. there exist more metrics and measures, you should learn to use them correctly with respect to the solution,dataset and other factors. - Do read up on the meaning and use of different metrics. there exist more metrics and measures, you should learn to use them correctly with respect to the solution,dataset and other factors.
- [F1 score](https://en.wikipedia.org/wiki/F1_score) and [Log Loss](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html) are the metrics for this challenge - [F1 score](https://en.wikipedia.org/wiki/F1_score) and [Log Loss](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html) are the metrics for this challenge
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
precision = precision_score(y_val,y_pred,average='micro') precision = precision_score(y_val,y_pred,average='micro')
recall = recall_score(y_val,y_pred,average='micro') recall = recall_score(y_val,y_pred,average='micro')
accuracy = accuracy_score(y_val,y_pred) accuracy = accuracy_score(y_val,y_pred)
f1 = f1_score(y_val,y_pred,average='macro') f1 = f1_score(y_val,y_pred,average='macro')
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
print("Accuracy of the model is :" ,accuracy) print("Accuracy of the model is :" ,accuracy)
print("Recall of the model is :" ,recall) print("Recall of the model is :" ,recall)
print("Precision of the model is :" ,precision) print("Precision of the model is :" ,precision)
print("F1 score of the model is :" ,f1) print("F1 score of the model is :" ,f1)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Here are some of the images predicted correctly by your model.Cheers! Here are some of the images predicted correctly by your model.Cheers!
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import matplotlib.pyplot as plt import matplotlib.pyplot as plt
def correctly_predicted(): def correctly_predicted():
correct_pred = np.where(y_pred == y_val) correct_pred = np.where(y_pred == y_val)
fig = plt.figure() fig = plt.figure()
for i in range(1,10): for i in range(1,10):
img = X_val.iloc[i,:] img = X_val.iloc[i,:]
img = np.array(img).reshape(28,28) img = np.array(img).reshape(28,28)
fig.add_subplot(3,3,i) fig.add_subplot(3,3,i)
plt.imshow(img,cmap='gray') plt.imshow(img,cmap='gray')
plt.show() plt.show()
correctly_predicted() correctly_predicted()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Testing Phase 😅 # Testing Phase 😅
We are almost done. We trained and validated on the training data. Now its the time to predict on test set and make a submission. We are almost done. We trained and validated on the training data. Now its the time to predict on test set and make a submission.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Load Test Set ## Load Test Set
Load the test data on which final submission is to be made. Load the test data on which final submission is to be made.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
final_test_path = "data/test.csv" final_test_path = "data/test.csv"
final_test = pd.read_csv(final_test_path,header=None) final_test = pd.read_csv(final_test_path,header=None)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Predict Test Set ## Predict Test Set
Predict on the test set and you are all set to make the submission ! Predict on the test set and you are all set to make the submission !
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
submission = classifier.predict(final_test) submission = classifier.predict(final_test)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Save the prediction to csv ## Save the prediction to csv
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
submission = pd.DataFrame(submission) submission = pd.DataFrame(submission)
submission.to_csv('submission.csv',header=['label'],index=False) submission.to_csv('submission.csv',header=['label'],index=False)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### 🚧 Note : ### 🚧 Note :
- Do take a look at the submission format. - Do take a look at the submission format.
- The submission file should contain a header. - The submission file should contain a header.
- Follow all submission guidelines strictly to avoid inconvenience. - Follow all submission guidelines strictly to avoid inconvenience.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## To download the generated csv in colab run the below command ## To download the generated csv in colab run the below command
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
try: try:
from google.colab import files from google.colab import files
files.download('submission.csv') files.download('submission.csv')
except: except:
print("only in Colab") print("only in Colab")
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### Well Done! 👍 We are all set to make a submission and see your name on leaderborad. Let navigate to [challenge page](https://www.aicrowd.com/challenges/mnist) and make one. ### Well Done! 👍 We are all set to make a submission and see your name on leaderborad. Let navigate to [challenge page](https://www.aicrowd.com/challenges/mnist) and make one.
......
baselines/MNIST_baseline.ipynb
+ 4
− 24
View file @ f17971c9
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
![AIcrowd-Logo](https://raw.githubusercontent.com/AIcrowd/AIcrowd/master/app/assets/images/misc/aicrowd-horizontal.png) ![AIcrowd-Logo](https://raw.githubusercontent.com/AIcrowd/AIcrowd/master/app/assets/images/misc/aicrowd-horizontal.png)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Getting Started Code for [MNIST Educational Challenge](https://www.aicrowd.com/challenges/mnist) # Getting Started Code for [MNIST Educational Challenge](https://www.aicrowd.com/challenges/mnist)
#### Author : Ayush Shivani #### Author : Ayush Shivani
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## To open this notebook on Google Computing platform Colab, click below! ## To open this notebook on Google Computing platform Colab, click below!
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ayushshivani/aicrowd_educational_baselines/blob/master/MNIST_baseline.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ayushshivani/aicrowd_educational_baselines/blob/master/MNIST_baseline.ipynb)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Download Necessary Packages 📚 ## Download Necessary Packages 📚
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import sys import sys
!pip install numpy !pip install numpy
!pip install pandas !pip install pandas
!pip install scikit-learn !pip install scikit-learn
!pip install matplotlib !pip install matplotlib
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Download Data ## Download Data
The first step is to download out train test data. We will be training a model on the train data and make predictions on test data. We submit our predictions. The first step is to download out train test data. We will be training a model on the train data and make predictions on test data. We submit our predictions.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
#Donwload the datasets #Donwload the datasets
!rm -rf data !rm -rf data
!mkdir data !mkdir data
!wget https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/mnist/v0.1/test.zip !wget https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/mnist/v0.1/test.zip
!wget https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/mnist/v0.1/train.zip !wget https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/mnist/v0.1/train.zip
!unzip test.zip !unzip test.zip
!unzip test.zip
!mv test.csv data/test.csv !mv test.csv data/test.csv
!mv train.csv data/train.csv !mv train.csv data/train.csv
!rm test.zip train.zip !rm test.zip train.zip
``` ```
%% Output
--2020-05-17 00:00:14-- https://s3.eu-central-1.wasabisys.com/aicrowd-practice-challenges/public/mnist/v0.1/test.zip
Resolving s3.eu-central-1.wasabisys.com (s3.eu-central-1.wasabisys.com)... 130.117.252.16, 130.117.252.12, 130.117.252.13, ...
Connecting to s3.eu-central-1.wasabisys.com (s3.eu-central-1.wasabisys.com)|130.117.252.16|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2202644 (2.1M) [application/zip]
Saving to: ‘test.zip’
test.zip 100%[===================>] 2.10M 64.3KB/s in 29s
2020-05-17 00:00:44 (74.3 KB/s) - ‘test.zip’ saved [2202644/2202644]
Archive: test.zip
inflating: test.csv
rm: cannot remove 'train.zip': No such file or directory
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Import packages ## Import packages
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import pandas as pd import pandas as pd
import numpy as np import numpy as np
from sklearn.model_selection import train_test_split from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC from sklearn.svm import SVC
from sklearn.metrics import f1_score,precision_score,recall_score,accuracy_score from sklearn.metrics import f1_score,precision_score,recall_score,accuracy_score
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Load Data ## Load Data
- We use pandas 🐼 library to load our data. - We use pandas 🐼 library to load our data.
- Pandas loads the data into dataframes and facilitates us to analyse the data. - Pandas loads the data into dataframes and facilitates us to analyse the data.
- Learn more about it [here](https://www.tutorialspoint.com/python_data_science/python_pandas.htm) 🤓 - Learn more about it [here](https://www.tutorialspoint.com/python_data_science/python_pandas.htm) 🤓
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
all_data_path = "data/train.csv" #path where data is stored all_data_path = "data/train.csv" #path where data is stored
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
all_data = pd.read_csv(all_data_path,header=None) #load data in dataframe using pandas all_data = pd.read_csv(all_data_path,header=None) #load data in dataframe using pandas
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Visualize the data 👀 ## Visualize the data 👀
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
all_data.head() all_data.head()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
You can see the columns goes from 0 to 784, where columns from 1 to 784 denotes pixel values each between 0-255 and the first column i.e. 0th is the digit it represents between 0-9. You can see the columns goes from 0 to 784, where columns from 1 to 784 denotes pixel values each between 0-255 and the first column i.e. 0th is the digit it represents between 0-9.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Split Data into Train and Validation 🔪 ## Split Data into Train and Validation 🔪
- The next step is to think of a way to test how well our model is performing. we cannot use the test data given as it does not contain the data labels for us to verify. - The next step is to think of a way to test how well our model is performing. we cannot use the test data given as it does not contain the data labels for us to verify.
- The workaround this is to split the given training data into training and validation. Typically validation sets give us an idea of how our model will perform on unforeseen data. it is like holding back a chunk of data while training our model and then using it to for the purpose of testing. it is a standard way to fine-tune hyperparameters in a model. - The workaround this is to split the given training data into training and validation. Typically validation sets give us an idea of how our model will perform on unforeseen data. it is like holding back a chunk of data while training our model and then using it to for the purpose of testing. it is a standard way to fine-tune hyperparameters in a model.
- There are multiple ways to split a dataset into validation and training sets. following are two popular ways to go about it, [k-fold](https://machinelearningmastery.com/k-fold-cross-validation/), [leave one out](https://en.wikipedia.org/wiki/Cross-validation_statistics). 🧐 - There are multiple ways to split a dataset into validation and training sets. following are two popular ways to go about it, [k-fold](https://machinelearningmastery.com/k-fold-cross-validation/), [leave one out](https://en.wikipedia.org/wiki/Cross-validation_statistics). 🧐
- Validation sets are also used to avoid your model from [overfitting](https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/) on the train dataset. - Validation sets are also used to avoid your model from [overfitting](https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/) on the train dataset.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
X_train, X_val= train_test_split(all_data, test_size=0.2, random_state=42) X_train, X_val= train_test_split(all_data, test_size=0.2, random_state=42)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
- We have decided to split the data with 20 % as validation and 80 % as training. - We have decided to split the data with 20 % as validation and 80 % as training.
- To learn more about the train_test_split function [click here](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html). 🧐 - To learn more about the train_test_split function [click here](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html). 🧐
- This is of course the simplest way to validate your model by simply taking a random chunk of the train set and setting it aside solely for the purpose of testing our train model on unseen data. as mentioned in the previous block, you can experiment 🔬 with and choose more sophisticated techniques and make your model better. - This is of course the simplest way to validate your model by simply taking a random chunk of the train set and setting it aside solely for the purpose of testing our train model on unseen data. as mentioned in the previous block, you can experiment 🔬 with and choose more sophisticated techniques and make your model better.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
- Now, since we have our data splitted into train and validation sets, we need to get the corresponding labels separated from the data. - Now, since we have our data splitted into train and validation sets, we need to get the corresponding labels separated from the data.
- with this step we are all set move to the next step with a prepared dataset. - with this step we are all set move to the next step with a prepared dataset.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
X_train,y_train = X_train.iloc[:,1:],X_train.iloc[:,0] X_train,y_train = X_train.iloc[:,1:],X_train.iloc[:,0]
X_val,y_val = X_val.iloc[:,1:],X_val.iloc[:,0] X_val,y_val = X_val.iloc[:,1:],X_val.iloc[:,0]
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# TRAINING PHASE 🏋️ # TRAINING PHASE 🏋️
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Define the Model ## Define the Model
- We have fixed our data and now we are ready to train our model. - We have fixed our data and now we are ready to train our model.
- There are a ton of classifiers to choose from some being [Logistic Regression](https://towardsdatascience.com/logistic-regression-detailed-overview-46c4da4303bc), [SVM](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Random Forests](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Decision Trees](https://towardsdatascience.com/decision-trees-in-machine-learning-641b9c4e8052), etc.🧐 - There are a ton of classifiers to choose from some being [Logistic Regression](https://towardsdatascience.com/logistic-regression-detailed-overview-46c4da4303bc), [SVM](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Random Forests](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Decision Trees](https://towardsdatascience.com/decision-trees-in-machine-learning-641b9c4e8052), etc.🧐
- Remember that there are no hard-laid rules here. you can mix and match classifiers, it is advisable to read up on the numerous techniques and choose the best fit for your solution , experimentation is the key. - Remember that there are no hard-laid rules here. you can mix and match classifiers, it is advisable to read up on the numerous techniques and choose the best fit for your solution , experimentation is the key.
- A good model does not depend solely on the classifier but also on the features you choose. So make sure to analyse and understand your data well and move forward with a clear view of the problem at hand. you can gain important insight from [here](https://towardsdatascience.com/the-5-feature-selection-algorithms-every-data-scientist-need-to-know-3a6b566efd2).🧐 - A good model does not depend solely on the classifier but also on the features you choose. So make sure to analyse and understand your data well and move forward with a clear view of the problem at hand. you can gain important insight from [here](https://towardsdatascience.com/the-5-feature-selection-algorithms-every-data-scientist-need-to-know-3a6b566efd2).🧐
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
classifier = LogisticRegression(solver = 'lbfgs',multi_class='auto',max_iter=10) classifier = LogisticRegression(solver = 'lbfgs',multi_class='auto',max_iter=10)
# from sklearn.svm import SVC # from sklearn.svm import SVC
# classifier = SVC(gamma='auto') # classifier = SVC(gamma='auto')
# from sklearn import tree # from sklearn import tree
# classifier = tree.DecisionTreeClassifier() # classifier = tree.DecisionTreeClassifier()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
- To start you off, We have used a basic [Support Vector Machines](https://scikit-learn.org/stable/modules/svm.html#classification) classifier here. - To start you off, We have used a basic [Support Vector Machines](https://scikit-learn.org/stable/modules/svm.html#classification) classifier here.
- But you can tune parameters and increase the performance. To see the list of parameters visit [here](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html). - But you can tune parameters and increase the performance. To see the list of parameters visit [here](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html).
- Do keep in mind there exist sophisticated techniques for everything, the key as quoted earlier is to search them and experiment to fit your implementation. - Do keep in mind there exist sophisticated techniques for everything, the key as quoted earlier is to search them and experiment to fit your implementation.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
To read more about other sklearn classifiers visit [here 🧐](https://scikit-learn.org/stable/supervised_learning.html). Try and use other classifiers to see how the performance of your model changes. Try using [Logistic Regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) or [MLP](http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html) and compare how the performance changes. To read more about other sklearn classifiers visit [here 🧐](https://scikit-learn.org/stable/supervised_learning.html). Try and use other classifiers to see how the performance of your model changes. Try using [Logistic Regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) or [MLP](http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html) and compare how the performance changes.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Train the Model ## Train the Model
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
classifier.fit(X_train, y_train) classifier.fit(X_train, y_train)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Got a warning! Dont worry, its just beacuse the number of iteration is very less(defined in the classifier in the above cell).Increase the number of iterations and see if the warning vanishes.Do remember increasing iterations also increases the running time.( Hint: max_iter=500) Got a warning! Dont worry, its just beacuse the number of iteration is very less(defined in the classifier in the above cell).Increase the number of iterations and see if the warning vanishes.Do remember increasing iterations also increases the running time.( Hint: max_iter=500)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Validation Phase 🤔 # Validation Phase 🤔
Wonder how well your model learned! Lets check it. Wonder how well your model learned! Lets check it.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Predict on Validation ## Predict on Validation
Now we predict using our trained model on the validation set we created and evaluate our model on unforeseen data. Now we predict using our trained model on the validation set we created and evaluate our model on unforeseen data.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
y_pred = classifier.predict(X_val) y_pred = classifier.predict(X_val)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Evaluate the Performance ## Evaluate the Performance
- We have used basic metrics to quantify the performance of our model. - We have used basic metrics to quantify the performance of our model.
- This is a crucial step, you should reason out the metrics and take hints to improve aspects of your model. - This is a crucial step, you should reason out the metrics and take hints to improve aspects of your model.
- Do read up on the meaning and use of different metrics. there exist more metrics and measures, you should learn to use them correctly with respect to the solution,dataset and other factors. - Do read up on the meaning and use of different metrics. there exist more metrics and measures, you should learn to use them correctly with respect to the solution,dataset and other factors.
- [F1 score](https://en.wikipedia.org/wiki/F1_score) and [Log Loss](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html) are the metrics for this challenge - [F1 score](https://en.wikipedia.org/wiki/F1_score) and [Log Loss](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html) are the metrics for this challenge
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
precision = precision_score(y_val,y_pred,average='micro') precision = precision_score(y_val,y_pred,average='micro')
recall = recall_score(y_val,y_pred,average='micro') recall = recall_score(y_val,y_pred,average='micro')
accuracy = accuracy_score(y_val,y_pred) accuracy = accuracy_score(y_val,y_pred)
f1 = f1_score(y_val,y_pred,average='macro') f1 = f1_score(y_val,y_pred,average='macro')
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
print("Accuracy of the model is :" ,accuracy) print("Accuracy of the model is :" ,accuracy)
print("Recall of the model is :" ,recall) print("Recall of the model is :" ,recall)
print("Precision of the model is :" ,precision) print("Precision of the model is :" ,precision)
print("F1 score of the model is :" ,f1) print("F1 score of the model is :" ,f1)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Here are some of the images predicted correctly by your model.Cheers! Here are some of the images predicted correctly by your model.Cheers!
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import matplotlib.pyplot as plt import matplotlib.pyplot as plt
def correctly_predicted(): def correctly_predicted():
correct_pred = np.where(y_pred == y_val) correct_pred = np.where(y_pred == y_val)
fig = plt.figure() fig = plt.figure()
for i in range(1,10): for i in range(1,10):
img = X_val.iloc[i,:] img = X_val.iloc[i,:]
img = np.array(img).reshape(28,28) img = np.array(img).reshape(28,28)
fig.add_subplot(3,3,i) fig.add_subplot(3,3,i)
plt.imshow(img,cmap='gray') plt.imshow(img,cmap='gray')
plt.show() plt.show()
correctly_predicted() correctly_predicted()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# Testing Phase 😅 # Testing Phase 😅
We are almost done. We trained and validated on the training data. Now its the time to predict on test set and make a submission. We are almost done. We trained and validated on the training data. Now its the time to predict on test set and make a submission.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Load Test Set ## Load Test Set
Load the test data on which final submission is to be made. Load the test data on which final submission is to be made.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
final_test_path = "data/test.csv" final_test_path = "data/test.csv"
final_test = pd.read_csv(final_test_path,header=None) final_test = pd.read_csv(final_test_path,header=None)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Predict Test Set ## Predict Test Set
Predict on the test set and you are all set to make the submission ! Predict on the test set and you are all set to make the submission !
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
submission = classifier.predict(final_test) submission = classifier.predict(final_test)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Save the prediction to csv ## Save the prediction to csv
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
submission = pd.DataFrame(submission) submission = pd.DataFrame(submission)
submission.to_csv('submission.csv',header=['label'],index=False) submission.to_csv('submission.csv',header=['label'],index=False)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### 🚧 Note : ### 🚧 Note :
- Do take a look at the submission format. - Do take a look at the submission format.
- The submission file should contain a header. - The submission file should contain a header.
- Follow all submission guidelines strictly to avoid inconvenience. - Follow all submission guidelines strictly to avoid inconvenience.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## To download the generated csv in colab run the below command ## To download the generated csv in colab run the below command
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
try: try:
from google.colab import files from google.colab import files
files.download('submission.csv') files.download('submission.csv')
except: except:
print("only in Colab") print("only in Colab")
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### Well Done! 👍 We are all set to make a submission and see your name on leaderborad. Let navigate to [challenge page](https://www.aicrowd.com/challenges/mnist) and make one. ### Well Done! 👍 We are all set to make a submission and see your name on leaderborad. Let navigate to [challenge page](https://www.aicrowd.com/challenges/mnist) and make one.
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment