PatrickBue
GitHub
Родитель
1b7daaccb0
Коммит
84ce55c86e
|
|
@ -8,6 +8,7 @@
|
|||
* [Which problems can be solved using image classification, and which ones cannot](#which-problems-can-be-solved-using-image-classification)
|
||||
* Data
|
||||
* [How many images are required to train a model?](#how-many-images-are-required-to-train-a-model)
|
||||
* [How to collect a large set of images?](#how-to-collect-a-large-set-of-images)
|
||||
* [How to annotate images?](#how-to-annotate-images)
|
||||
* [How to split into training and test images?](#How-to-split-into-training-and-test-images)
|
||||
* [How to design a good test set?](#how-to-design-a-good-test-set)
|
||||
|
|
@ -15,6 +16,7 @@
|
|||
* Training
|
||||
* [How to improve accuracy or inference speed?](#how-to-improve-accuracy-or-inference-speed)
|
||||
|
||||
|
||||
### How does the technology work?
|
||||
State-of-the-art image classification methods such as used in this repository are based on Convolutional Neural Networks (CNN). CNNs are a special group of Deep Learning approaches shown to work well on image data. The key is to use CNNs which were already trained on millions of images (the ImageNet dataset) and to fine-tune these pre-trained CNNs using a potentially much smaller custom dataset. This is the approach also taken in this repository. The web is full of introductions to these conceptions, such as [link](https://towardsdatascience.com/simple-introduction-to-convolutional-neural-networks-cdf8d3077bac).
|
||||
|
||||
|
|
@ -29,6 +31,22 @@ This depends heavily on the complexity of the problem. For example, if the objec
|
|||
In practice, we have seen good results using 100 images for each class or sometime less. The only way to find out how many images are required, is by training the model using increasing number of images, while observing how the accuracy improves (while keeping the test set fixed). Once accuracy improvements become small, this would indicate that more training images are not required.
|
||||
|
||||
|
||||
### How to collect a large set of images?
|
||||
Collecting a sufficiently large number of annotated images for training and testing can be difficult. One way to over-come this problem is to scrape images from the Internet. For example, see below (left image) the Bing Image Search results for the query "tshirt striped". As expected, most images indeed are striped t-shirts, and the few incorrect or ambiguous images (such as column 1, row 1; or column 3, row 2) can be identified and removed easily. Rather than manually downloading images from Bing Image Search, the [Cognitive Services Bing Image Search API](https://www.microsoft.com/cognitive-services/en-us/bing-image-search-api) (right image) can be used instead.
|
||||
|
||||
|Bing Image Search | Cognitive Services Image Search|
|
||||
|:-------------------------:|:-------------------------:|
|
||||
|<img src="media/bing_search_striped.jpg" alt="alt text" width="400"/> | <img src="media/bing_image_search_api.jpg" alt="alt text" width="400"/>|
|
||||
|
||||
To generate a large and diverse dataset, multiple queries should be used. For example 7\*3 = 21 queries can by synthesized using all combinations of 7 clothing items {blouse, hoodie, pullover, sweater, shirt, tshirt, vest} and 3 attributes {striped, dotted, leopard}. Downloading the top 50 images per query would then lead to a maximum of 21*50=1050 images.
|
||||
|
||||
|
||||
|
||||
Some of the downloaded images will be exact or near duplicates (e.g. differ just by image resolution or jpg artifacts) and should be removed so that the training and test split do not contain the same images. This can be achieved using a hashing-based approach which works in two steps: (i) first, the hash string is computed for all images; (ii) only images are kept with a hash string which has not yet been seen. All other images are discarded. We found the *dhash* approach in the Python library *imagehash* and described in this [blog](http://www.hackerfactor.com/blog/index.php?/archives/529-Kind-of-Like-That.html) to perform well, with the parameter `hash_size` set to 16. It is OK to incorrectly remove some non-duplicates images, as long as the majority of the real duplicates get removed.
|
||||
|
||||
|
||||
|
||||
|
||||
### How to annotate images?
|
||||
Consistency is key. For example, occluded objects should either be always annotated, or never. Furthermore, ambiguous images should be removed, eg if it is unclear to a human eye if an image shows a lemon or a tennis ball. Ensuring consistency is difficult especially if multiple people are involved, and hence our recommendation is that only a single person, the one who trains the AI model, annotates all images. This has the added benefit of gaining a better understanding of the images and of the complexity of the classification task.
|
||||
|
||||
|
|
|
|||
Двоичный файл не отображается.
|
После Ширина: | Высота: | Размер: 314 KiB |
Двоичный файл не отображается.
|
После Ширина: | Высота: | Размер: 173 KiB |
|
|
@ -0,0 +1,184 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<i>Copyright (c) Microsoft Corporation. All rights reserved.</i>\n",
|
||||
"\n",
|
||||
"<i>Licensed under the MIT License.</i>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Image annotation UI"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Open-source annotation tools for object detection and for image segmentation exist, however for image classification we were not able to find a good program. Hence this notebook provides a simple UI to label images. Each image can be annotated with one or multiple classes, or marked as \"Exclude\" to indicate that the image should not be used for model training or evaluation. \n",
|
||||
"\n",
|
||||
"Note that, for single class annotation tasks, one does not need any UI but can instead simply drag-and-drop images into separate folder for the respective classes. \n",
|
||||
"\n",
|
||||
"See the [FAQ.md](..\\FAQ.md) for a brief discussion on how to scrape images from the internet."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Ensure edits to libraries are loaded and plotting is shown in the notebook.\n",
|
||||
"%reload_ext autoreload\n",
|
||||
"%autoreload 2\n",
|
||||
"%matplotlib inline"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os, sys\n",
|
||||
"sys.path.append(\"../\")\n",
|
||||
"from utils_ic.anno_utils import AnnotationWidget\n",
|
||||
"from utils_ic.datasets import unzip_url, Urls"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Set parameters: location of the images to annotate, and path where to save the annotations. Here `unzip_url` is used to download example data if not already present, and set the path."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"tags": [
|
||||
"parameters"
|
||||
]
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Using images in directory: C:\\Users\\pabuehle\\Desktop\\ComputerVisionBestPractices\\image_classification\\data\\fridgeObjects\\can.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"IM_DIR = os.path.join((unzip_url(Urls.fridge_objects_path, exist_ok=True)), 'can')\n",
|
||||
"ANNO_PATH = \"cvbp_ic_annotation.txt\"\n",
|
||||
"print(f\"Using images in directory: {IM_DIR}.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Start the UI. Check the \"Allow multi-class labeling\" box to allow for images to be annotated with multiple classes. When in doubt what the annotation for an image should be, or for any other reason (e.g. blur or over-exposure), mark an image as \"EXCLUDE\". All annotations are saved to (and loaded from) a pandas dataframe with path specified in `anno_path`. \n",
|
||||
"\n",
|
||||
"<center>\n",
|
||||
"<img src=\"media/anno_ui.jpg\" style=\"width: 600px;\"/>\n",
|
||||
"<i>Annotation UI example</i>\n",
|
||||
"</center>"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Loading existing annotation from cvbp_ic_annotation.txt.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"data": {
|
||||
"application/vnd.jupyter.widget-view+json": {
|
||||
"model_id": "492cae0bb3e340babb935f688660d11f",
|
||||
"version_major": 2,
|
||||
"version_minor": 0
|
||||
},
|
||||
"text/plain": [
|
||||
"Tab(children=(VBox(children=(HBox(children=(Button(description='Previous', layout=Layout(width='80px'), style=…"
|
||||
]
|
||||
},
|
||||
"metadata": {},
|
||||
"output_type": "display_data"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"w_anno_ui = AnnotationWidget(\n",
|
||||
" labels = [\"can\", \"carton\", \"milk_bottle\", \"water_bottle\"],\n",
|
||||
" im_dir = IM_DIR,\n",
|
||||
" anno_path = ANNO_PATH,\n",
|
||||
" im_filenames = None #Set to None to annotate all images in IM_DIR\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"display(w_anno_ui.show())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Below is an example how to create a fast.ai ImageList object using the ground truth annotations generated by the AnnotationWidget. Note that fast.ai does not support the exclude flag, hence we remove these images before calling fast.ai's `from_df()` and `label_from_df()` functions.\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"import pandas as pd\n",
|
||||
"from fastai.vision import ImageList,ImageDataBunch\n",
|
||||
"\n",
|
||||
"# Load annotation, discard excluded images, and convert to format fastai expects\n",
|
||||
"data = []\n",
|
||||
"with open(ANNO_PATH,'r') as f:\n",
|
||||
" for line in f.readlines()[1:]:\n",
|
||||
" vec = line.strip().split(\"\\t\")\n",
|
||||
" exclude = vec[1]==\"True\"\n",
|
||||
" if not exclude and len(vec)>2:\n",
|
||||
" data.append((vec[0], vec[2]))\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame(data, columns = [\"name\", \"label\"])\n",
|
||||
"display(df)\n",
|
||||
"\n",
|
||||
"data = (ImageList.from_df(path=IM_DIR, df = df)\n",
|
||||
" .split_by_rand_pct(valid_pct=0.5)\n",
|
||||
" .label_from_df(cols='label', label_delim=','))\n",
|
||||
"```"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python (cvbp)",
|
||||
"language": "python",
|
||||
"name": "cvbp"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.6.8"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
Двоичный файл не отображается.
|
После Ширина: | Высота: | Размер: 67 KiB |
|
|
@ -0,0 +1,85 @@
|
|||
#!/usr/bin/env python
|
||||
# coding: utf-8
|
||||
|
||||
# <i>Copyright (c) Microsoft Corporation. All rights reserved.</i>
|
||||
#
|
||||
# <i>Licensed under the MIT License.</i>
|
||||
|
||||
# # Image annotation UI
|
||||
|
||||
# Open-source annotation tools for object detection and for image segmentation exist, however for image classification we were not able to find a good program. Hence this notebook provides a simple UI to label images. Each image can be annotated with one or multiple classes, or marked as "Exclude" to indicate that the image should not be used for model training or evaluation.
|
||||
#
|
||||
# Note that, for single class annotation tasks, one does not need any UI but can instead simply drag-and-drop images into separate folder for the respective classes.
|
||||
#
|
||||
# See the [FAQ.md](..\FAQ.md) for a brief discussion on how to scrape images from the internet.
|
||||
|
||||
# In[1]:
|
||||
|
||||
|
||||
# Ensure edits to libraries are loaded and plotting is shown in the notebook.
|
||||
get_ipython().run_line_magic('reload_ext', 'autoreload')
|
||||
get_ipython().run_line_magic('autoreload', '2')
|
||||
get_ipython().run_line_magic('matplotlib', 'inline')
|
||||
|
||||
|
||||
# In[2]:
|
||||
|
||||
|
||||
import os, sys
|
||||
sys.path.append("../")
|
||||
from utils_ic.anno_utils import AnnotationWidget
|
||||
from utils_ic.datasets import unzip_url, Urls
|
||||
|
||||
|
||||
# Set parameters: location of the images to annotate, and path where to save the annotations. Here `unzip_url` is used to download example data if not already present, and set the path.
|
||||
|
||||
# In[3]:
|
||||
|
||||
|
||||
IM_DIR = os.path.join((unzip_url(Urls.fridge_objects_path, exist_ok=True)), 'can')
|
||||
ANNO_PATH = "cvbp_ic_annotation.txt"
|
||||
print(f"Using images in directory: {IM_DIR}.")
|
||||
|
||||
|
||||
# Start the UI. Check the "Allow multi-class labeling" box to allow for images to be annotated with multiple classes. When in doubt what the annotation for an image should be, or for any other reason (e.g. blur or over-exposure), mark an image as "EXCLUDE". All annotations are saved to (and loaded from) a pandas dataframe with path specified in `anno_path`.
|
||||
#
|
||||
# <center>
|
||||
# <img src="media/anno_ui.jpg" style="width: 600px;"/>
|
||||
# <i>Annotation UI example</i>
|
||||
# </center>
|
||||
|
||||
# In[4]:
|
||||
|
||||
|
||||
w_anno_ui = AnnotationWidget(
|
||||
labels = ["can", "carton", "milk_bottle", "water_bottle"],
|
||||
im_dir = IM_DIR,
|
||||
anno_path = ANNO_PATH,
|
||||
im_filenames = None #Set to None to annotate all images in IM_DIR
|
||||
)
|
||||
|
||||
display(w_anno_ui.show())
|
||||
|
||||
|
||||
# Below is an example how to create a fast.ai ImageList object using the ground truth annotations generated by the AnnotationWidget. Note that fast.ai does not support the exclude flag, hence we remove these images before calling fast.ai's `from_df()` and `label_from_df()` functions.
|
||||
#
|
||||
# ```python
|
||||
# import pandas as pd
|
||||
# from fastai.vision import ImageList,ImageDataBunch
|
||||
#
|
||||
# # Load annotation, discard excluded images, and convert to format fastai expects
|
||||
# data = []
|
||||
# with open(ANNO_PATH,'r') as f:
|
||||
# for line in f.readlines()[1:]:
|
||||
# vec = line.strip().split("\t")
|
||||
# exclude = vec[1]=="True"
|
||||
# if not exclude and len(vec)>2:
|
||||
# data.append((vec[0], vec[2]))
|
||||
#
|
||||
# df = pd.DataFrame(data, columns = ["name", "label"])
|
||||
# display(df)
|
||||
#
|
||||
# data = (ImageList.from_df(path=IM_DIR, df = df)
|
||||
# .split_by_rand_pct(valid_pct=0.5)
|
||||
# .label_from_df(cols='label', label_delim=','))
|
||||
# ```
|
||||
|
|
@ -35,6 +35,8 @@ def notebooks():
|
|||
"02_training_accuracy_vs_speed": os.path.join(
|
||||
folder_notebooks, "02_training_accuracy_vs_speed.ipynb"
|
||||
),
|
||||
"10_image_annotation": os.path.join(
|
||||
folder_notebooks, "10_image_annotation.ipynb"),
|
||||
"11_exploring_hyperparameters": os.path.join(
|
||||
folder_notebooks, "11_exploring_hyperparameters.ipynb"
|
||||
),
|
||||
|
|
|
|||
|
|
@ -0,0 +1,55 @@
|
|||
import os
|
||||
import numpy as np
|
||||
from pathlib import Path
|
||||
from PIL import Image
|
||||
import pytest
|
||||
from utils_ic.common import data_path, get_files_in_directory, ic_root_path, im_height, im_width, im_width_height
|
||||
|
||||
|
||||
|
||||
def test_ic_root_path():
|
||||
s = ic_root_path()
|
||||
assert isinstance(s, str) and s != ""
|
||||
|
||||
|
||||
def test_data_path():
|
||||
s = data_path()
|
||||
assert isinstance(s, str) and s != ""
|
||||
|
||||
|
||||
def test_im_width(tiny_ic_data_path):
|
||||
im_path = Path(tiny_ic_data_path)/"can"/"1.jpg"
|
||||
assert (
|
||||
im_width(im_path) == 499
|
||||
), "Expected image width of 499, but got {}".format(im_width(im_path))
|
||||
im = np.zeros((100, 50))
|
||||
assert im_width(im) == 50, "Expected image width of 50, but got ".format(
|
||||
im_width(im)
|
||||
)
|
||||
|
||||
|
||||
def test_im_height(tiny_ic_data_path):
|
||||
im_path = Path(tiny_ic_data_path)/"can"/"1.jpg"
|
||||
assert (
|
||||
im_height(im_path) == 665
|
||||
), "Expected image height of 665, but got ".format(im_width(60))
|
||||
im = np.zeros((100, 50))
|
||||
assert (
|
||||
im_height(im) == 100
|
||||
), "Expected image height of 100, but got ".format(im_width(im))
|
||||
|
||||
|
||||
def test_im_width_height(tiny_ic_data_path):
|
||||
im_path = Path(tiny_ic_data_path)/"can"/"1.jpg"
|
||||
w, h = im_width_height(im_path)
|
||||
assert w == 499 and h == 665
|
||||
im = np.zeros((100, 50))
|
||||
w, h = im_width_height(im)
|
||||
assert w == 50 and h == 100
|
||||
|
||||
|
||||
def test_get_files_in_directory(tiny_ic_data_path):
|
||||
im_dir = os.path.join(tiny_ic_data_path, "can")
|
||||
assert len(get_files_in_directory(im_dir)) == 22
|
||||
assert len(get_files_in_directory(im_dir, suffixes=[".jpg"])) == 22
|
||||
assert len(get_files_in_directory(im_dir, suffixes=[".nonsense"])) == 0
|
||||
|
|
@ -1,16 +1,15 @@
|
|||
# This test is based on the test suite implemented for Recommenders project
|
||||
# https://github.com/Microsoft/Recommenders/tree/master/tests
|
||||
|
||||
|
||||
import glob
|
||||
import os
|
||||
import glob
|
||||
import papermill as pm
|
||||
import shutil
|
||||
|
||||
# Unless manually modified, python3 should be
|
||||
# the name of the current jupyter kernel
|
||||
# that runs on the activated conda environment
|
||||
KERNEL_NAME = "python3"
|
||||
KERNEL_NAME = "cvbp"
|
||||
OUTPUT_NOTEBOOK = "output.ipynb"
|
||||
|
||||
|
||||
|
|
@ -52,6 +51,19 @@ def test_02_notebook_run(notebooks, tiny_ic_data_path):
|
|||
)
|
||||
|
||||
|
||||
def test_10_notebook_run(notebooks, tiny_ic_data_path):
|
||||
notebook_path = notebooks["10_image_annotation"]
|
||||
pm.execute_notebook(
|
||||
notebook_path,
|
||||
OUTPUT_NOTEBOOK,
|
||||
parameters=dict(
|
||||
PM_VERSION=pm.__version__,
|
||||
IM_DIR=os.path.join(tiny_ic_data_path, "can"),
|
||||
),
|
||||
kernel_name=KERNEL_NAME,
|
||||
)
|
||||
|
||||
|
||||
def test_11_notebook_run(notebooks, tiny_ic_data_path):
|
||||
notebook_path = notebooks["11_exploring_hyperparameters"]
|
||||
pm.execute_notebook(
|
||||
|
|
@ -104,3 +116,4 @@ def skip_test_deploy_1_notebook_run(notebooks, tiny_ic_data_path):
|
|||
shutil.rmtree(os.path.join(os.getcwd(), "azureml-models"))
|
||||
shutil.rmtree(os.path.join(os.getcwd(), "models"))
|
||||
shutil.rmtree(os.path.join(os.getcwd(), "outputs"))
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,265 @@
|
|||
# Copyright (c) Microsoft Corporation. All rights reserved.
|
||||
# Licensed under the MIT License.
|
||||
|
||||
import os
|
||||
from ipywidgets import widgets, Layout, IntSlider
|
||||
import pandas as pd
|
||||
from utils_ic.common import im_width, im_height, get_files_in_directory
|
||||
|
||||
|
||||
class AnnotationWidget(object):
|
||||
IM_WIDTH = 500 # pixels
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
labels: list,
|
||||
im_dir: str,
|
||||
anno_path: str,
|
||||
im_filenames: list = None,
|
||||
):
|
||||
"""Widget class to annotate images.
|
||||
|
||||
Args:
|
||||
labels: List of abel names, e.g. ["bird", "car", "plane"].
|
||||
im_dir: Directory containing the images to be annotated.
|
||||
anno_path: path where to write annotations to, and (if exists) load annotations from.
|
||||
im_fnames: List of image filenames. If set to None, then will auto-detect all images in the provided image directory.
|
||||
"""
|
||||
self.labels = labels
|
||||
self.im_dir = im_dir
|
||||
self.anno_path = anno_path
|
||||
self.im_filenames = im_filenames
|
||||
|
||||
# Init
|
||||
self.vis_image_index = 0
|
||||
self.label_to_id = {s: i for i, s in enumerate(self.labels)}
|
||||
if not im_filenames:
|
||||
self.im_filenames = [os.path.basename(s) for s in get_files_in_directory(
|
||||
im_dir,
|
||||
suffixes=(
|
||||
".jpg",
|
||||
".jpeg",
|
||||
".tif",
|
||||
".tiff",
|
||||
".gif",
|
||||
".giff",
|
||||
".png",
|
||||
".bmp",
|
||||
),
|
||||
)]
|
||||
assert len(self.im_filenames) > 0, f"Not a single image specified or found in directory {im_dir}."
|
||||
|
||||
# Initialize empty annotations and load previous annotations if file exist
|
||||
self.annos = pd.DataFrame()
|
||||
for im_filename in self.im_filenames:
|
||||
if im_filename not in self.annos:
|
||||
self.annos[im_filename] = pd.Series(
|
||||
{"exclude": False, "labels": []}
|
||||
)
|
||||
if os.path.exists(self.anno_path):
|
||||
print(f"Loading existing annotation from {self.anno_path}.")
|
||||
with open(self.anno_path,'r') as f:
|
||||
for line in f.readlines()[1:]:
|
||||
vec = line.strip().split("\t")
|
||||
im_filename = vec[0]
|
||||
self.annos[im_filename].exclude = vec[1]=="True"
|
||||
if len(vec)>2:
|
||||
self.annos[im_filename].labels = vec[2].split(',')
|
||||
|
||||
|
||||
# Create UI and "start" widget
|
||||
self._create_ui()
|
||||
|
||||
def show(self):
|
||||
return self.ui
|
||||
|
||||
def update_ui(self):
|
||||
im_filename = self.im_filenames[self.vis_image_index]
|
||||
im_path = os.path.join(self.im_dir, im_filename)
|
||||
|
||||
# Update the image and info
|
||||
self.w_img.value = open(im_path, "rb").read()
|
||||
self.w_filename.value = im_filename
|
||||
self.w_path.value = self.im_dir
|
||||
|
||||
# Fix the width of the image widget and adjust the height
|
||||
self.w_img.layout.height = (
|
||||
f"{int(self.IM_WIDTH * (im_height(im_path)/im_width(im_path)))}px"
|
||||
)
|
||||
|
||||
# Update annotations
|
||||
self.exclude_widget.value = self.annos[im_filename].exclude
|
||||
for w in self.label_widgets:
|
||||
w.value = False
|
||||
for label in self.annos[im_filename].labels:
|
||||
label_id = self.label_to_id[label]
|
||||
self.label_widgets[label_id].value = True
|
||||
|
||||
def _create_ui(self):
|
||||
"""Create and initialize widgets"""
|
||||
# ------------
|
||||
# Callbacks + logic
|
||||
# ------------
|
||||
def skip_image():
|
||||
"""Return true if image should be skipped, and false otherwise."""
|
||||
# See if UI-checkbox to skip images is checked
|
||||
if not self.w_skip_annotated.value:
|
||||
return False
|
||||
|
||||
# Stop skipping if image index is out of bounds
|
||||
if (
|
||||
self.vis_image_index <= 0
|
||||
or self.vis_image_index >= len(self.im_filenames) - 1
|
||||
):
|
||||
return False
|
||||
|
||||
# Skip if image has annotation
|
||||
im_filename = self.im_filenames[self.vis_image_index]
|
||||
labels = self.annos[im_filename].labels
|
||||
exclude = self.annos[im_filename].exclude
|
||||
if exclude or len(labels) > 0:
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
def button_pressed(obj):
|
||||
"""Next / previous image button callback."""
|
||||
# Find next/previous image. Variable step is -1 or +1 depending on which button was pressed.
|
||||
step = int(obj.value)
|
||||
self.vis_image_index += step
|
||||
while skip_image():
|
||||
self.vis_image_index += step
|
||||
|
||||
self.vis_image_index = min(
|
||||
max(self.vis_image_index, 0), len(self.im_filenames) - 1
|
||||
)
|
||||
self.w_image_slider.value = self.vis_image_index
|
||||
self.update_ui()
|
||||
|
||||
def slider_changed(obj):
|
||||
"""Image slider callback.
|
||||
Need to wrap in try statement to avoid errors when slider value is not a number.
|
||||
"""
|
||||
try:
|
||||
self.vis_image_index = int(obj["new"]["value"])
|
||||
self.update_ui()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def anno_changed(obj):
|
||||
"""Label checkbox callback.
|
||||
Update annotation file and write to disk
|
||||
"""
|
||||
# Test if call is coming from the user having clicked on a checkbox to change its state,
|
||||
# rather than a change of state when e.g. the checkbox value was updated programatically. This is a bit
|
||||
# of hack, but necessary since widgets.Checkbox() does not support a on_click() callback or similar.
|
||||
if "new" in obj and isinstance(obj["new"], dict) and len(obj["new"]) == 0:
|
||||
# If single-label annotation then unset all checkboxes except the one which the user just clicked
|
||||
if not self.w_multi_class.value:
|
||||
for w in self.label_widgets:
|
||||
if w.description != obj["owner"].description:
|
||||
w.value = False
|
||||
|
||||
# Update annotation object
|
||||
im_filename = self.im_filenames[self.vis_image_index]
|
||||
self.annos[im_filename].labels = [
|
||||
w.description for w in self.label_widgets if w.value
|
||||
]
|
||||
self.annos[im_filename].exclude = self.exclude_widget.value
|
||||
|
||||
# Write to disk as tab-separated file.
|
||||
with open(self.anno_path,'w') as f:
|
||||
f.write("{}\t{}\t{}\n".format("IM_FILENAME", "EXCLUDE", "LABELS"))
|
||||
for k,v in self.annos.items():
|
||||
if v.labels != [] or v.exclude:
|
||||
f.write("{}\t{}\t{}\n".format(k, v.exclude, ",".join(v.labels)))
|
||||
|
||||
|
||||
# ------------
|
||||
# UI - image + controls (left side)
|
||||
# ------------
|
||||
w_next_image_button = widgets.Button(description="Next")
|
||||
w_next_image_button.value = "1"
|
||||
w_next_image_button.layout = Layout(width="80px")
|
||||
w_next_image_button.on_click(button_pressed)
|
||||
w_previous_image_button = widgets.Button(description="Previous")
|
||||
w_previous_image_button.value = "-1"
|
||||
w_previous_image_button.layout = Layout(width="80px")
|
||||
w_previous_image_button.on_click(button_pressed)
|
||||
|
||||
self.w_filename = widgets.Text(
|
||||
value="", description="Name:", layout=Layout(width="200px")
|
||||
)
|
||||
self.w_path = widgets.Text(
|
||||
value="", description="Path:", layout=Layout(width="200px")
|
||||
)
|
||||
self.w_image_slider = IntSlider(
|
||||
min=0,
|
||||
max=len(self.im_filenames) - 1,
|
||||
step=1,
|
||||
value=self.vis_image_index,
|
||||
continuous_update=False,
|
||||
)
|
||||
self.w_image_slider.observe(slider_changed)
|
||||
self.w_img = widgets.Image()
|
||||
self.w_img.layout.width = f"{self.IM_WIDTH}px"
|
||||
|
||||
w_header = widgets.HBox(
|
||||
children=[
|
||||
w_previous_image_button,
|
||||
w_next_image_button,
|
||||
self.w_image_slider,
|
||||
self.w_filename,
|
||||
self.w_path,
|
||||
]
|
||||
)
|
||||
|
||||
# ------------
|
||||
# UI - info (right side)
|
||||
# ------------
|
||||
# Options widgets
|
||||
self.w_skip_annotated = widgets.Checkbox(
|
||||
value=False, description="Skip annotated images."
|
||||
)
|
||||
self.w_multi_class = widgets.Checkbox(
|
||||
value=False, description="Allow multi-class labeling"
|
||||
)
|
||||
|
||||
# Label checkboxes widgets
|
||||
self.exclude_widget = widgets.Checkbox(
|
||||
value=False, description="EXCLUDE IMAGE"
|
||||
)
|
||||
self.exclude_widget.observe(anno_changed)
|
||||
self.label_widgets = [
|
||||
widgets.Checkbox(value=False, description=label)
|
||||
for label in self.labels
|
||||
]
|
||||
for label_widget in self.label_widgets:
|
||||
label_widget.observe(anno_changed)
|
||||
|
||||
# Combine UIs into tab widget
|
||||
w_info = widgets.VBox(
|
||||
children=[
|
||||
widgets.HTML(value="Options:"),
|
||||
self.w_skip_annotated,
|
||||
self.w_multi_class,
|
||||
widgets.HTML(value="Annotations:"),
|
||||
self.exclude_widget,
|
||||
*self.label_widgets,
|
||||
]
|
||||
)
|
||||
w_info.layout.padding = "20px"
|
||||
self.ui = widgets.Tab(
|
||||
children=[
|
||||
widgets.VBox(
|
||||
children=[
|
||||
w_header,
|
||||
widgets.HBox(children=[self.w_img, w_info]),
|
||||
]
|
||||
)
|
||||
]
|
||||
)
|
||||
self.ui.set_title(0, "Annotator")
|
||||
|
||||
# Fill UI with content
|
||||
self.update_ui()
|
||||
|
|
@ -1,5 +1,8 @@
|
|||
import os
|
||||
import numpy as np
|
||||
from pathlib import Path
|
||||
from PIL import Image
|
||||
from typing import Union, Tuple, List
|
||||
|
||||
|
||||
def ic_root_path() -> Path:
|
||||
|
|
@ -12,3 +15,59 @@ def data_path() -> Path:
|
|||
return os.path.realpath(
|
||||
os.path.join(os.path.dirname(__file__), os.pardir, "data")
|
||||
)
|
||||
|
||||
|
||||
def im_width(input: Union[str, np.array]) -> int:
|
||||
"""Returns the width of an image.
|
||||
Args:
|
||||
input: Image path or image as numpy array.
|
||||
Return:
|
||||
Image width.
|
||||
"""
|
||||
return im_width_height(input)[0]
|
||||
|
||||
|
||||
def im_height(input: Union[str, np.array]) -> int:
|
||||
"""Returns the height of an image.
|
||||
Args:
|
||||
input: Image path or image as numpy array.
|
||||
Return:
|
||||
Image height.
|
||||
"""
|
||||
return im_width_height(input)[1]
|
||||
|
||||
|
||||
def im_width_height(input: Union[str, np.array]) -> Tuple[int, int]:
|
||||
"""Returns the width and height of an image.
|
||||
Args:
|
||||
input: Image path or image as numpy array.
|
||||
Return:
|
||||
Tuple of ints (width,height).
|
||||
"""
|
||||
if isinstance(input, str) or isinstance(input, Path):
|
||||
width, height = Image.open(
|
||||
input
|
||||
).size # this is fast since it does not load the full image
|
||||
else:
|
||||
width, height = (input.shape[1], input.shape[0])
|
||||
return width, height
|
||||
|
||||
|
||||
def get_files_in_directory(
|
||||
directory: str, suffixes: List[str] = None
|
||||
) -> List[str]:
|
||||
"""Returns all filenames in a directory which optionally match one of multiple suffixes.
|
||||
Args:
|
||||
directory: directory to scan for files.
|
||||
suffixes: only keep the filenames which ends with one of the suffixes (e.g. suffixes = [".jpg", ".png", ".gif"]).
|
||||
Return:
|
||||
List of filenames
|
||||
"""
|
||||
if not os.path.exists(directory):
|
||||
raise Exception(f"Directory '{directory}' does not exist.")
|
||||
filenames = [str(p) for p in Path(directory).iterdir() if p.is_file()]
|
||||
if suffixes and suffixes != "":
|
||||
filenames = [
|
||||
s for s in filenames if s.lower().endswith(tuple(suffixes))
|
||||
]
|
||||
return filenames
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче