Commit f4e7e883 authored by ashivani's avatar ashivani
Browse files

added Readme

parents
# To be filled in later with an consistent contribution guide
- Q : Who writes that ?
\ No newline at end of file
>**🛠 Contribute:** Found a typo? Or any other change in the description that you would like to see? Please consider sending us a pull request in the [public repo of the challenge here](https://gitlab.aicrowd.com/aicrowd/practice-challenges/aicrowd_JIGSAW_challenge).
# 🕵️ Introduction
<img src="https://i.imgur.com/0tDcVLp.png" width=600 height = 300>
Remember the summer vacation nostalgia of sitting down to solve a giant 200 piece puzzle?
At some point, all of us enjoyed and solved puzzles. The panic of a missing piece and the joy of completing puzzles was exciting!
This challenge will try to recreate the retro-puzzle solving with an AI twist!
Given a set of jumbled images, can you sort them in the correct order to solve the puzzle and form the complete picture?
The mission, if you choose to accept, is to classify and sort the many jumbled images in the correct order and submit a final solution with the correct order number.
</br>Understand with code! Here is [`getting started code`](https://discourse.aicrowd.com/t/baseline-jigsaw/3695) for you.`😄`
# 💾 Dataset
The dataset contains puzzle pieces for an images, inside the folder with name as the image id in `puzzles.tar.gz`. The height and widhth of original image can be found in the `metadata.csv` . There are total `2500` such folders, with puzzle pieces of the respective image ids.
# 📁 Files
Following files are available in the `resources` section:
* `puzzles.tar.gz`: A tar.gz file which when extracted, has 2500 folders (where folder names are the puzzle-id), and each of the folders contain the individual puzzle pieces comprising this puzzle as PNG files.
* `sample_submission.tar.gz`: A tar.gz file with 2500 randomly reconstructed images of each of the puzzles in the test set. The naming convention for each of the files inside this tar is `<puzzle_id>.jpg`.
* `metadata.csv` : A file containing the width and height of each of the puzzles in the test set.
# 🚀 Submission
- Recreate the original images with the puzzle pieces given in each folder for an image.
- Name the image as `{image_id}.jpg`
- Create a `tar.gz` file containing all the recreated images.
- For eg
```
submission.tar.gz
├── 0.jpg
|── 1.jpg
└── ....
```
- `Sample_submission.tar.gz` can be found in resources section.
**Make your first submission [here](https://www.aicrowd.com/challenges/jigsaw/submissions/new) 🚀 !!**
# 🖊 Evaluation Criteria
This challenge uses the [SSIM score](https://en.wikipedia.org/wiki/Structural_similarity) as the primary evaluation metric and the [Mean Squared Error](https://en.wikipedia.org/wiki/Mean_squared_error).
For all the puzzles in the dataset, the individual scores are computed by comparing the submitted reconstructed image in reference to the original image. The overall submission score is the mean SSIM and MSE scores across the whole data
**The score of only 60% of the test data will be revealed during the competition.**
# 🔗 Links
* 💪 Challenge Page: [https://www.aicrowd.com/challenges/jigsaw](https://www.aicrowd.com/challenges/jigsaw)
* 🗣️ Discussion Forum: [https://www.aicrowd.com/challenges/jigsaw/discussion](https://www.aicrowd.com/challenges/jigsaw/discussion)
* 🏆 Leaderboard: [https://www.aicrowd.com/challenges/jigsaw/leaderboards](https://www.aicrowd.com/challenges/jigsaw/leaderboards)
# 📱 Contact
- [Sharada Mohanty](mailto:mohanty@aicrowd.com)
\ No newline at end of file
---
challenge_name: jigsaw
evaluation_repo: git@gitlab.aicrowd.com:aicrowd/practice-challenges/aicrowd_jigsaw_challenge_evaluator.git
data_url: https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/jigsaw/v0.1/
official_baseline: JIGSAW_baseline.ipynb
authors:
- name: Sharada Mohanty
email: mohanty@aicrowd.com
version: '0.1'
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![AIcrowd-Logo](https://raw.githubusercontent.com/AIcrowd/AIcrowd/master/app/assets/images/misc/aicrowd-horizontal.png)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Getting Started Code for [JIGSAW Challenge](www.aicrowd.com/challenges/jigsaw) on AIcrowd\n",
"#### Author : Sanjay Pokkali"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Download Necessary Packages 📚"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install numpy\n",
"!pip install pandas\n",
"!pip install scikit-learn\n",
"!pip install textdistance"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Download Data\n",
"The first step is to download out train test data. We will be training a model on the train data and make predictions on test data. We submit our predictions.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"!rm -rf data\n",
"!mkdir data\n",
"!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/cptcha/v0.1/train.tar.gz\n",
"!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/cptcha/v0.1/test.tar.gz\n",
"!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/cptcha/v0.1/train_info.csv\n",
"!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/cptcha/v0.1/test_info.csv\n",
" \n",
"!tar -xvzf train.tar.gz\n",
"!tar -xvzf test.tar.gz\n",
"!mv train data/train\n",
"!mv test data/test\n",
"!mv train_info.csv data/train_info.csv\n",
"!mv test_info.csv data/test_info.csv"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Import packages"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"from sklearn.model_selection import train_test_split\n",
"from PIL import Image\n",
"import glob\n",
"import tempfile\n",
"import os \n",
"import random\n",
"import tqdm\n",
"import tarfile\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load Data\n",
"- We use pandas 🐼 library to load our data. \n",
"- Pandas loads the data into dataframes and facilitates us to analyse the data. \n",
"- Learn more about it [here](https://www.tutorialspoint.com/python_data_science/python_pandas.htm) 🤓"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"train_info_path = \"data/train_info.csv\"\n",
"test_info_path = \"data/test_info.csv\"\n",
"\n",
"train_images_path = \"data/train/\"\n",
"test_images_path = \"data/test/\"\n",
"train_info = pd.read_csv(train_info_path)\n",
"test_info = pd.read_csv(test_info_path)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Visualize the images👀"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def plot_image(img_path):\n",
" img = cv2.imread(img_path)\n",
"# print(\"Shape of the captcha \",img.shape)\n",
" plt.imshow(img)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"fig=plt.figure(figsize=(20,20))\n",
"columns = 3\n",
"rows = 3\n",
"for i in range(1, columns*rows +1):\n",
" img = train_images_path + train_info['filename'][i]\n",
" fig.add_subplot(rows, columns, i)\n",
" plot_image(img)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Split Data into Train and Validation 🔪\n",
"- The next step is to think of a way to test how well our model is performing. we cannot use the test data given as it does not contain the data labels for us to verify. \n",
"- The workaround this is to split the given training data into training and validation. Typically validation sets give us an idea of how our model will perform on unforeseen data. it is like holding back a chunk of data while training our model and then using it to for the purpose of testing. it is a standard way to fine-tune hyperparameters in a model. \n",
"- There are multiple ways to split a dataset into validation and training sets. following are two popular ways to go about it, [k-fold](https://machinelearningmastery.com/k-fold-cross-validation/), [leave one out](https://en.wikipedia.org/wiki/Cross-validation_statistics). 🧐\n",
"- Validation sets are also used to avoid your model from [overfitting](https://machinelearningmastery.com/overfitting-and-underfitting-with-machine-learning-algorithms/) on the train dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"X_train, X_val= train_test_split(train_info, test_size=0.2, random_state=42) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- We have decided to split the data with 20 % as validation and 80 % as training. \n",
"- To learn more about the train_test_split function [click here](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html). 🧐 \n",
"- This is of course the simplest way to validate your model by simply taking a random chunk of the train set and setting it aside solely for the purpose of testing our train model on unseen data. as mentioned in the previous block, you can experiment 🔬 with and choose more sophisticated techniques and make your model better."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- Now, since we have our data splitted into train and validation sets, we need to get the corresponding labels separated from the data. \n",
"- with this step we are all set move to the next step with a prepared dataset."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# TRAINING PHASE 🏋️"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**We will use PyTesseract, an Optical Character Recognition library to recognize the characters in the test captcha directly and make a submission in this notebook.But lest see its performace on the train set.**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Using PyTesseract on Training Set"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"labels = []\n",
"all_filenames = []\n",
"\n",
"for index,rows in train_info.iterrows():\n",
"\n",
" i = rows['filename']\n",
" img_path = train_images_path + i\n",
" label = pytesseract.image_to_string(Image.open(img_path))\n",
" #Removing garbage characters\n",
" label = label.replace(\"\\x0c\",\"\")\n",
" label = label.replace(\"\\n\",\"\")\n",
" labels.append(label)\n",
" all_filenames.append(i)\n",
" print(f'{str(index)+\"/\" + str(train_info.shape[0])}\\r',end=\"\")\n",
" \n",
" \n",
"\n",
"labels = np.asarray(labels)\n",
"all_filenames = np.asarray(all_filenames)\n",
"\n",
"\n",
"submission = pd.DataFrame()\n",
"submission['filename'] = all_filenames\n",
"submission['label'] = labels\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Evaluate the Performance\n",
"\n",
"**Here for evaluation mean over normalised [Levenshtein Similarity Score](https://en.wikipedia.org/wiki/Levenshtein_distance) will be used to test the efficiency of the model.**\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def cal_lshtein_score(s_true,s_pred):\n",
" if type(s_pred) == type(1.0):\n",
" return 0\n",
" score = textdistance.levenshtein.normalized_similarity(s_true,s_pred) \n",
" return score\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"lst_scores = []\n",
"for idx in range(0,len(train_info)):\n",
" lst_scores.append(cal_lshtein_score(train_info['label'][idx],submission['label'][idx]))\n",
"\n",
"mean_lst_score = np.mean(lst_scores)\n",
"\n",
"print(\"The mean of normalised Levenshtein Similarity score is \" ,mean_lst_score)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Testing Phase\n",
"\n",
"## Generate Output for Test set"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"labels = []\n",
"all_filenames = []\n",
"\n",
"for index,rows in test_info.iterrows():\n",
"\n",
" i = rows['filename']\n",
" img_path = test_images_path + i\n",
" label = pytesseract.image_to_string(Image.open(img_path))\n",
" #Removing garbage characters\n",
" label = label.replace(\"\\x0c\",\"\")\n",
" label = label.replace(\"\\n\",\"\")\n",
" labels.append(label)\n",
" all_filenames.append(i)\n",
" print(f'{str(index)+\"/\" + str(test_info.shape[0])}\\r',end=\"\")\n",
"\n",
"labels = np.asarray(labels)\n",
"all_filenames = np.asarray(all_filenames)\n",
"\n",
"\n",
"submission_df = pd.DataFrame()\n",
"submission_df['filename'] = all_filenames\n",
"submission_df['label'] = labels\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Save the prediction to csv"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"submission_df.to_csv('submission.csv', index=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 🚧 Note : \n",
"- Do take a look at the submission format. \n",
"- The submission file should contain a header. \n",
"- Follow all submission guidelines strictly to avoid inconvenience."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## To download the generated csv in colab run the below command"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"try:\n",
" from google.colab import files\n",
" files.download('submission.csv') \n",
"except:\n",
" print(\"Option Only avilable in Google Colab\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Well Done! 👍 We are all set to make a submission and see your name on leaderborad. Lets navigate to [challenge page](https://www.aicrowd.com/challenges/CPTCHA) and make one."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"![AIcrowd-Logo](https://raw.githubusercontent.com/AIcrowd/AIcrowd/master/app/assets/images/misc/aicrowd-horizontal.png)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Getting Started Code for [JIGSAW Challenge](www.aicrowd.com/challenges/jigsaw) on AIcrowd\n",
"#### Author : Sanjay Pokkali"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Download Necessary Packages 📚"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install numpy\n",
"!pip install pandas\n",
"!pip install scikit-learn\n",
"!pip install textdistance"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Download Data\n",
"The first step is to download out train test data. We will be training a model on the train data and make predictions on test data. We submit our predictions.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"!rm -rf data\n",
"!mkdir data\n",
"!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/cptcha/v0.1/train.tar.gz\n",
"!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/cptcha/v0.1/test.tar.gz\n",
"!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/cptcha/v0.1/train_info.csv\n",
"!wget https://datasets.aicrowd.com/default/aicrowd-practice-challenges/public/cptcha/v0.1/test_info.csv\n",
" \n",
"!tar -xvzf train.tar.gz\n",
"!tar -xvzf test.tar.gz\n",
"!mv train data/train\n",
"!mv test data/test\n",
"!mv train_info.csv data/train_info.csv\n",
"!mv test_info.csv data/test_info.csv"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Import packages"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import numpy as np\n",
"from sklearn.model_selection import train_test_split\n",
"from PIL import Image\n",
"import glob\n",
"import tempfile\n",
"import os \n",
"import random\n",
"import tqdm\n",
"import tarfile\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load Data\n",
"- We use pandas 🐼 library to load our data. \n",
"- Pandas loads the data into dataframes and facilitates us to analyse the data. \n",
"- Learn more about it [here](https://www.tutorialspoint.com/python_data_science/python_pandas.htm) 🤓"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"train_info_path = \"data/train_info.csv\"\n",
"test_info_path = \"data/test_info.csv\"\n",
"\n",
"train_images_path = \"data/train/\"\n",
"test_images_path = \"data/test/\"\n",
"train_info = pd.read_csv(train_info_path)\n",
"test_info = pd.read_csv(test_info_path)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Visualize the images👀"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def plot_image(img_path):\n",
" img = cv2.imread(img_path)\n",
"# print(\"Shape of the captcha \",img.shape)\n",
" plt.imshow(img)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"fig=plt.figure(figsize=(20,20))\n",
"columns = 3\n",
"rows = 3\n",
"for i in range(1, columns*rows +1):\n",
" img = train_images_path + train_info['filename'][i]\n",
" fig.add_subplot(rows, columns, i)\n",
" plot_image(img)\n",
"plt.show()"
]