{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "AUTODRI_baseline.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VN0fF-CL9b8U",
        "colab_type": "text"
      },
      "source": [
        "![AIcrowd-Logo](https://raw.githubusercontent.com/AIcrowd/AIcrowd/master/app/assets/images/misc/aicrowd-horizontal.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "BbEmUIM8yVuO"
      },
      "source": [
        "# Baseline for [AUTODRI Challenge](https://www.aicrowd.com/challenges/autodri) on AIcrowd\n",
        "#### Author : Ayush Shivani"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "dszDet1yyVub"
      },
      "source": [
        "## Download Necessary Packages"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "oAJIrtbZyVue",
        "colab": {}
      },
      "source": [
        "import sys\n",
        "!pip install numpy\n",
        "!pip install pandas\n",
        "!pip install scikit-learn \n",
        "!pip install matplotlib tqdm "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "DTYfZoSkyVuw"
      },
      "source": [
        "## Download data\n",
        "The first step is to download the training data and the test data\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "fuY6PfZtyVuy",
        "colab": {}
      },
      "source": [
        "# #Donwload the datasets\n",
        "!rm -rf data/\n",
        "!mkdir data/\n",
        "\n",
        "!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/autodri/v0.1/train.zip\n",
        "!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/autodri/v0.1/test.zip\n",
        "!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/autodri/v0.1/val.zip\n",
        "!unzip train.zip  \n",
        "!unzip test.zip \n",
        "!unzip val.zip\n",
        "!mv train data/train\n",
        "!mv test data/test\n",
        "!mv val data/val"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "B3XYXsD-yVvG",
        "colab": {}
      },
      "source": [
        "## Now the data is available at the following locations:\n",
        "\n",
        "TRAINING_IMAGES_FOLDER = \"data/train/cameraFront\"\n",
        "TRAINING_LABELS_PATH = \"data/train/train.csv\"\n",
        "TESTING_LABELS_PATH = \"data/test/test.csv\"\n",
        "TESTING_IMAGES_FOLDER = \"data/test/cameraFront\"\n",
        "# For this baseline, we will only be using the front camera angle of the car just for demonstration purpose. For actual one should try and see the best combination of all the angles"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "iz17HhTHyVvP"
      },
      "source": [
        "\n",
        "## Import packages"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "E6DV5-etyVvS",
        "colab": {}
      },
      "source": [
        "import os\n",
        "import tqdm\n",
        "\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.neural_network import MLPRegressor\n",
        "from sklearn.metrics import mean_squared_error,mean_absolute_error\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "\n",
        "from PIL import Image \n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "zYZYR-vYyVve"
      },
      "source": [
        "## Load Data\n",
        "We use PIL library to load our images. Here we are creating our array where our input features are the mean colours and output features are the rotations along the x axis."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Rx6_Y33zyVvg",
        "colab": {}
      },
      "source": [
        "training_labels_df = pd.read_csv(TRAINING_LABELS_PATH)\n",
        "\n",
        "def pre_process_data_X(image):\n",
        "    \"\"\"\n",
        "    This file takes a loaded image and returns a particular \n",
        "    representation of the data point\n",
        "    \n",
        "    \n",
        "    NOTE: This current baseline implements a **very** silly approach\n",
        "    of representing every image by the mean RGB values for every image.\n",
        "    \n",
        "    You are encourage to try to alternate representations of the data,\n",
        "    or figure out how to learn the best representation from the data ;)\n",
        "    \"\"\"\n",
        "    im_array = np.array(im)\n",
        "    mean_rgb = im_array.mean(axis=(0, 1))\n",
        "    return mean_rgb\n",
        "\n",
        "\n",
        "ALL_DATA = []\n",
        "\n",
        "for _idx, row in tqdm.tqdm(training_labels_df.iterrows(), total=training_labels_df.shape[0]):\n",
        "    filepath = os.path.join(\n",
        "        TRAINING_IMAGES_FOLDER,\n",
        "        row.filename\n",
        "    )\n",
        "    im = Image.open(filepath)\n",
        "    \n",
        "    data_X = pre_process_data_X(im)\n",
        "    data_Y = [row.canSteering]\n",
        "    \n",
        "    ALL_DATA.append((data_X, data_Y))\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "sezNtTkq3hxT"
      },
      "source": [
        "## Exploratory Data Analysis\n",
        "We now see the kind of images the dataset contains to get a better idea."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "JQhY7rNe3idL",
        "colab": {}
      },
      "source": [
        "plt.figure(figsize=(20,20))\n",
        "for i in range(16):\n",
        "  filename,xRot = training_labels_df.iloc[i]\n",
        "  filepath = os.path.join(\n",
        "        TRAINING_IMAGES_FOLDER,\n",
        "        filename\n",
        "    )\n",
        "  im = Image.open(filepath)\n",
        "  plt.subplot(4,4,i+1)\n",
        "  plt.axis('off')\n",
        "  plt.title(\"canSteering: %.3f\"%(xRot))\n",
        "  plt.imshow(im)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "TYvINkWeyVvy"
      },
      "source": [
        "## Split Data into Train and Validation\n",
        "We split the dataset into Training data and Validation datasets to help us test the generalizability of our models, and to ensure that we are not overfitting on the training set."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "GDfJrh7dyVvz",
        "colab": {}
      },
      "source": [
        "training_set, validation_set= train_test_split(ALL_DATA, test_size=0.2, random_state=42) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "XH_LMKnUyVv8"
      },
      "source": [
        "Here we have selected the size of the testing data to be 20% of the total data. You can change it and see what effect it has on the accuracies. To learn more about the train_test_split function [click here](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "cYpk-CZwyVv9"
      },
      "source": [
        "Now, since we have our data splitted into train and validation sets, we need to get the label separated from the data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "DB_uMnwayVwJ",
        "colab": {}
      },
      "source": [
        "X_train, y_train = zip(*training_set)\n",
        "X_val, y_val = zip(*validation_set)\n",
        "\n",
        "\n",
        "X_train = np.array(X_train)\n",
        "y_train = np.array(y_train)\n",
        "X_val = np.array(X_val)\n",
        "y_val = np.array(y_val)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "U0AnLiL-yVwY"
      },
      "source": [
        "## Define the Classifier\n",
        "Now we finally come to the juicy part. \n",
        "Now that all the data is all loaded and available nice, we can finally get to training the classifier. Here we use sklearn [`MLPRegressor`](https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html) to train our network. We can tune the hyper parameters based on cross validation scores "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "JYgnxy-gyVwa",
        "colab": {}
      },
      "source": [
        "model = MLPRegressor(hidden_layer_sizes=[10, 10], verbose=True)\n",
        "# NOTE : This is again silly hyper parameter instantiation of this problem,\n",
        "# and we encourage you to explore what works the best for you."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "mU7AjMzRyVwk"
      },
      "source": [
        "## Train the classifier"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "FC8WsiwfyVwl",
        "colab": {}
      },
      "source": [
        "model.fit(X_train, y_train)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "3EvltSxayVwt"
      },
      "source": [
        "## Predict on Validation\n",
        "Now we predict our trained classifier on the validation set and evaluate our model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "gSO5uNfJyVww",
        "colab": {}
      },
      "source": [
        "y_pred = model.predict(X_val)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "6MRB7XKtyVw2"
      },
      "source": [
        "## Evaluate the Performance\n",
        "We use the same metrics as that will be used for the test set.  \n",
        "[MAE](https://en.wikipedia.org/wiki/Mean_absolute_error) and [RMSE](https://www.statisticshowto.com/rmse/) are the metrics for this challenge"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "NR3ZYR_iyVw_",
        "colab": {}
      },
      "source": [
        "print('Mean Absolute Error:', mean_absolute_error(y_val, y_pred))  \n",
        "print('Mean Squared Error:', mean_squared_error(y_val, y_pred))  \n",
        "print('Root Mean Squared Error:', np.sqrt(mean_squared_error(y_val, y_pred)))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "CqX7U8fzyVxG"
      },
      "source": [
        "## Load Test Set\n",
        "Load the test data now"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "ccXpZaGUyVxI",
        "colab": {}
      },
      "source": [
        "import glob\n",
        "testing_labels_df = pd.read_csv(TESTING_LABELS_PATH)\n",
        "\n",
        "TEST_DATA = []\n",
        "TEST_FILENAMES = []\n",
        "for _idx, row in tqdm.tqdm(testing_labels_df.iterrows(), total=testing_labels_df.shape[0]):\n",
        "    filepath = os.path.join(\n",
        "        TESTING_IMAGES_FOLDER,\n",
        "        row.filename\n",
        "    )\n",
        "    print(filepath)\n",
        "    im = Image.open(filepath)\n",
        "    \n",
        "    data_X = pre_process_data_X(im)\n",
        "    TEST_DATA.append(data_X)\n",
        "    TEST_FILENAMES.append(row.filename)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "88XPXX4iyVxU"
      },
      "source": [
        "## Make predictions on the test set"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "D9RehB3-yVxX",
        "colab": {}
      },
      "source": [
        "test_predictions = model.predict(TEST_DATA)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_gcjeVrKLLgz",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "test_predictions.shape"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "ydEUtO8RyVxd",
        "colab": {}
      },
      "source": [
        "test_df = pd.DataFrame(test_predictions, columns=['canSteering'])\n",
        "test_df[\"filename\"] = TEST_FILENAMES"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UNqS7uoDJ2La",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "test_df.shape"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "rEFUNuY6yVxi"
      },
      "source": [
        "## Save the prediction to csv"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "QsMrHzYNyVxk",
        "colab": {}
      },
      "source": [
        "test_df.to_csv('submission.csv', index=False)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "gr9K9BZuyVxp"
      },
      "source": [
        "**Note**: Do take a look at the submission format.The submission file should contain the following header : `filename,xRot`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "yNpniOcWyVxs"
      },
      "source": [
        "## To download the generated csv in Google Colab run the below command"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "eNy4UTTkyVxt",
        "colab": {}
      },
      "source": [
        "from google.colab import files\n",
        "files.download('submission.csv')"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "-N1UT5tTyVx0"
      },
      "source": [
        "### Well Done! 👍 We are all set to make a submission and see your name on leaderborad. Lets navigate to [challenge page](https://www.aicrowd.com/challenges/autodri) and make one."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5DOM-WsV9Xur",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}