WINEQ_baseline.ipynb 13.7 KB
Newer Older
faixan's avatar
faixan committed
1 2 3 4 5 6 7 8 9 10 11 12 13
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![AIcrowd-Logo](https://raw.githubusercontent.com/AIcrowd/AIcrowd/master/app/assets/images/misc/aicrowd-horizontal.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
ashivani's avatar
ashivani committed
14
    "# Baseline for [WINEQ Educational Challenge](https://www.aicrowd.com/challenges/wineq) on AIcrowd\n",
faixan's avatar
faixan committed
15 16 17 18 19 20 21 22 23 24 25 26
    "#### Author : Faizan Farooq Khan"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download Necessary Packages"
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
27 28 29
   "execution_count": null,
   "metadata": {},
   "outputs": [],
faixan's avatar
faixan committed
30 31
   "source": [
    "import sys\n",
ashivani's avatar
ashivani committed
32 33 34
    "!pip install numpy\n",
    "!pip install pandas\n",
    "!pip install scikit-learn"
faixan's avatar
faixan committed
35 36 37 38 39 40 41 42 43 44 45 46
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download Data\n",
    "The first step is to download out train test data. We will be training a model on the train data and make predictions on test data. We submit our predictions\n"
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
47 48 49
   "execution_count": null,
   "metadata": {},
   "outputs": [],
faixan's avatar
faixan committed
50
   "source": [
ashivani's avatar
ashivani committed
51 52
    "!rm -rf data\n",
    "!mkdir data\n",
ashivani's avatar
ashivani committed
53 54
    "!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/wineq/v0.1/test.csv\n",
    "!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/wineq/v0.1/train.csv\n",
ashivani's avatar
ashivani committed
55 56
    "!mv train.csv data/train.csv\n",
    "!mv test.csv data/test.csv"
faixan's avatar
faixan committed
57 58 59 60 61 62 63 64 65 66 67 68
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Import packages"
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
69
   "execution_count": null,
faixan's avatar
faixan committed
70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.metrics import f1_score,precision_score,recall_score,accuracy_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Data\n",
    "- We use pandas 🐼 library to load our data.   \n",
    "- Pandas loads the data into dataframes and facilitates us to analyse the data.   \n",
    "- Learn more about it [here](https://www.tutorialspoint.com/python_data_science/python_pandas.htm) 🤓"
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
94
   "execution_count": null,
faixan's avatar
faixan committed
95 96 97
   "metadata": {},
   "outputs": [],
   "source": [
ashivani's avatar
ashivani committed
98
    "all_data_path = \"data/train.csv\" #path where data is stored"
faixan's avatar
faixan committed
99 100 101 102
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
103
   "execution_count": null,
faixan's avatar
faixan committed
104 105 106
   "metadata": {},
   "outputs": [],
   "source": [
ashivani's avatar
ashivani committed
107
    "all_data = pd.read_csv(all_data_path,header=None) #load data in dataframe using pandas"
faixan's avatar
faixan committed
108 109 110 111 112 113 114 115 116 117 118
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize the data 👀"
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
119
   "execution_count": null,
faixan's avatar
faixan committed
120 121 122
   "metadata": {},
   "outputs": [],
   "source": [
ashivani's avatar
ashivani committed
123
    "all_data.head()"
faixan's avatar
faixan committed
124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the dataset contains 12 columns,where columns 0-10 denotes different attributes of the wine the last column tells the quality of the wine from 1-10."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split Data into Train and Validation 🔪\n",
    "-  The next step is to think of a way to test how well our model is performing. we cannot use the test data given as it does not contain the data labels for us to verify.    \n",
    "- The workaround this is to split the given training data into training and validation. Typically validation sets give us an idea of how our model will perform on unforeseen data. it is like holding back a chunk of data while training our model and then using it to for the purpose of testing. it is a standard way to fine-tune hyperparameters in a model.    \n",
    "- There are multiple ways to split a dataset into validation and training sets. following are two popular ways to go about it, [k-fold](https://machinelearningmastery.com/k-fold-cross-validation/), [leave one out](https://en.wikipedia.org/wiki/Cross-validation_statistics). 🧐\n",
    "- Validation sets are also used to avoid your model from [overfitting](https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/) on the train dataset."
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
146
   "execution_count": null,
faixan's avatar
faixan committed
147 148 149
   "metadata": {},
   "outputs": [],
   "source": [
ashivani's avatar
ashivani committed
150
    "X_train, X_val= train_test_split(all_data, test_size=0.2, random_state=42) "
faixan's avatar
faixan committed
151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- We have decided to split the data with 20 % as validation and 80 % as training.  \n",
    "- To learn more about the train_test_split function [click here](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html). 🧐  \n",
    "- This is of course the simplest way to validate your model by simply taking a random chunk of the train set and setting it aside solely for the purpose of testing our train model on unseen data. as mentioned in the previous block, you can experiment 🔬 with and choose more sophisticated techniques and make your model better."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Now, since we have our data splitted into train and validation sets, we need to get the corresponding labels separated from the data.   \n",
    "- with this step we are all set move to the next step with a prepared dataset."
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
172
   "execution_count": null,
faixan's avatar
faixan committed
173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train,y_train = X_train.iloc[:,:-1],X_train.iloc[:,-1]\n",
    "X_val,y_val = X_val.iloc[:,:-1],X_val.iloc[:,-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TRAINING PHASE 🏋️"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the Model\n",
    "\n",
    "- We have fixed our data and now we are ready to train our model.   \n",
    "\n",
    "- There are a ton of classifiers to choose from some being [Logistic Regression](https://towardsdatascience.com/logistic-regression-detailed-overview-46c4da4303bc), [SVM](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Random Forests](https://towardsdatascience.com/support-vector-machine-introduction-to-machine-learning-algorithms-934a444fca47), [Decision Trees](https://towardsdatascience.com/decision-trees-in-machine-learning-641b9c4e8052), etc.🧐         \n",
    "\n",
    "- Remember that there are no hard-laid rules here. you can mix and match classifiers, it is advisable to read up on the numerous techniques and choose the best fit for your solution , experimentation is the key.     \n",
    "   \n",
    "- A good model does not depend solely on the classifier but also on the features you choose. So make sure to analyse and understand your data well and move forward with a clear view of the problem at hand.  you can gain important insight from [here](https://towardsdatascience.com/the-5-feature-selection-algorithms-every-data-scientist-need-to-know-3a6b566efd2).🧐         "
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
204
   "execution_count": null,
faixan's avatar
faixan committed
205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238
   "metadata": {},
   "outputs": [],
   "source": [
    "classifier = SVC(gamma='auto')\n",
    "\n",
    "#from sklearn.linear_model import LogisticRegression\n",
    "# classifier = LogisticRegression()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- To start you off, We have used a basic [Support Vector Machines](https://scikit-learn.org/stable/modules/svm.html#classification) classifier here.    \n",
    "- But you can tune parameters and increase the performance. To see the list of parameters visit [here](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html).   \n",
    "- Do keep in mind there exist sophisticated techniques for everything, the key as quoted earlier is to search them and experiment to fit your implementation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To read more about other sklearn classifiers visit [here 🧐](https://scikit-learn.org/stable/supervised_learning.html). Try and use other classifiers to see how the performance of your model changes. Try using [Logistic Regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) or [MLP](http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html) and compare how the performance changes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the classifier"
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
239 240 241
   "execution_count": null,
   "metadata": {},
   "outputs": [],
faixan's avatar
faixan committed
242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271
   "source": [
    "classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Got a warning! Dont worry, its just beacuse the number of iteration is very less(defined in the classifier in the above cell).Increase the number of iterations and see if the warning vanishes and also see how the performance changes.Do remember increasing iterations also increases the running time.( Hint: max_iter=500)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Validation Phase 🤔\n",
    "Wonder how well your model learned! Lets check it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predict on Validation\n",
    "\n",
    "Now we predict using our trained model on the validation set we created and evaluate our model on unforeseen data."
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
272
   "execution_count": null,
faixan's avatar
faixan committed
273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred = classifier.predict(X_val)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate the Performance\n",
    "\n",
    "- We have used basic metrics to quantify the performance of our model.  \n",
    "- This is a crucial step, you should reason out the metrics and take hints to improve aspects of your model.\n",
    "- Do read up on the meaning and use of different metrics. there exist more metrics and measures, you should learn to use them correctly with respect to the solution,dataset and other factors. \n",
    "- [F1 score](https://en.wikipedia.org/wiki/F1_score) is the metric for this challenge"
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
293 294 295
   "execution_count": null,
   "metadata": {},
   "outputs": [],
faixan's avatar
faixan committed
296 297 298 299 300 301 302 303 304
   "source": [
    "precision = precision_score(y_val,y_pred,average='micro')\n",
    "recall = recall_score(y_val,y_pred,average='micro')\n",
    "accuracy = accuracy_score(y_val,y_pred)\n",
    "f1 = f1_score(y_val,y_pred,average='macro')"
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
305 306 307
   "execution_count": null,
   "metadata": {},
   "outputs": [],
faixan's avatar
faixan committed
308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333
   "source": [
    "print(\"Accuracy of the model is :\" ,accuracy)\n",
    "print(\"Recall of the model is :\" ,recall)\n",
    "print(\"Precision of the model is :\" ,precision)\n",
    "print(\"F1 score of the model is :\" ,f1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Testing Phase 😅\n",
    "\n",
    "We are almost done. We trained and validated on the training data. Now its the time to predict on test set and make a submission."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Test Set\n",
    "Load the test data now"
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
334
   "execution_count": null,
faixan's avatar
faixan committed
335 336 337
   "metadata": {},
   "outputs": [],
   "source": [
ashivani's avatar
ashivani committed
338
    "final_test_path = \"data/test.csv\"\n",
faixan's avatar
faixan committed
339 340 341 342 343 344 345 346 347 348 349 350 351
    "final_test = pd.read_csv(final_test_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predict Test Set\n",
    "Time for the moment of truth! Predict on test set and time to make the submission."
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
352
   "execution_count": null,
faixan's avatar
faixan committed
353 354 355 356 357 358 359 360 361 362 363 364 365 366 367
   "metadata": {},
   "outputs": [],
   "source": [
    "submission = classifier.predict(final_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save the prediction to csv"
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
368
   "execution_count": null,
faixan's avatar
faixan committed
369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394
   "metadata": {},
   "outputs": [],
   "source": [
    "submission = pd.DataFrame(submission)\n",
    "submission.to_csv('submission.csv',header=['quality'],index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "🚧 Note :    \n",
    "- Do take a look at the submission format.   \n",
    "- The submission file should contain a header.   \n",
    "- Follow all submission guidelines strictly to avoid inconvenience."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## To download the generated csv in colab run the below command"
   ]
  },
  {
   "cell_type": "code",
ashivani's avatar
ashivani committed
395
   "execution_count": null,
faixan's avatar
faixan committed
396 397 398 399 400 401 402 403 404 405 406 407 408 409
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    from google.colab import files\n",
    "    files.download('submission.csv')\n",
    "except:\n",
    "    print('only in colab')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
ashivani's avatar
ashivani committed
410
    "### Well Done! 👍 We are all set to make a submission and see your name on leaderborad. Let navigate to [challenge page](https://www.aicrowd.com/challenges/wineq) and make one."
faixan's avatar
faixan committed
411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
ashivani's avatar
ashivani committed
430
   "version": "3.7.6"
faixan's avatar
faixan committed
431 432 433 434 435
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}