Object detection neural network for detecting a frying pan in images
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
Garrett Wilson 7c21cbf0d9 added Youtube link 1 year ago
google-images-download@6a3dd67d64 Initial commit 2 years ago
models@4298c3a3b5 Initial commit 2 years ago
.gitignore added negative images 1 year ago
.gitmodules cherry-pick from detect-laptop, reduce false positives 1 year ago
README.md added Youtube link 1 year ago
annotate.sh Initial commit 2 years ago
config.sh cherry-pick from detect-laptop, reduce false positives 1 year ago
delete_files.py higher-altitude shots 1 year ago
eval.sh Shrink images for dataset, multiple model options 2 years ago
export.sh Shrink images for dataset, multiple model options 2 years ago
export_nonlite.sh Added non-TF Lite export script 2 years ago
gen_sloth.sh Initial commit 2 years ago
google_image_config.json Initial commit 2 years ago
shrink_images.sh Initial commit 2 years ago
sloth.json higher-altitude shots 1 year ago
sloth2tf.py support snowy weather 1 year ago
sloth_common.py cherry-pick from detect-laptop, reduce false positives 1 year ago
sloth_config.py Initial commit 2 years ago
sloth_remove_deleted.py delete many files since too many 1 year ago
ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync.config Shrink images for dataset, multiple model options 2 years ago
ssd_mobilenet_v1_coco.config evaluate more 1 year ago
ssd_mobilenet_v1_ppn_shared_box_predictor_300x300_coco14_sync.config Shrink images for dataset, multiple model options 2 years ago
tf_label_map.pbtxt Initial commit 2 years ago
train.sh Shrink images for dataset, multiple model options 2 years ago

README.md

Detect Frying Pan

Use an object detection neural network to detect frying pans in images. The purpose of this is for use in an autonomous drone egg drop entry.

See the demo video on Youtube.

Instructions

Get this code:

git clone --recursive https://github.com/floft/detect-frying-pan
cd detect-frying-pan

Collect images for dataset

Download images from Google

sudo pacman -S python-selenium chromium
./google-images-download/google_images_download/google_images_download.py \
    -cf google_image_config.json

Delete all those you don't like from the google_images/ folder, e.g. if they're not black, have watermarks, are clipart, have a white background, have low depth of field, etc. Look for images that are mostly top-down mostly empty if possible.

Since there will likely be duplicates, you can remove some of these (if they're exact duplicates):

jdupes google_images/ -r -d # select which you want, e.g. 1 for all duplicates

Take your own images

Buy a skillet or frying pan and go take top-down pictures from a variety of angles with many different backgrounds and lighting conditions. Maybe with a variety of cameras too. Put them all in a my_images_large/ folder.

Shrink them. Outputs to my_images/:

./shrink_images.sh

Labeling images

Then, to label them in Sloth (see my Arch PKGBUILD):

./gen_sloth.sh
./annotate.sh # 'f' to label as frying pan, space for next, Ctrl+S to save

Convert TensorFlow {tftrain,tfvalid,tftest}.record files:

./sloth2tf.py

Get pre-trained TensorFlow networks

wget http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_coco_2018_01_28.tar.gz
wget http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18.tar.gz
wget http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_ppn_shared_box_predictor_300x300_coco14_sync_2018_07_03.tar.gz
for i in *.tar.gz; do tar xaf $i; done

Fix bug

To fix a bug, in models/research/object_detection/metrics/coco_tools.py change

results.dataset['categories'] = copy.deepcopy(self.dataset['categories'])

to

results.dataset['categories'] = self.dataset['categories']

Training

Uncomment the model you wish to train in config.sh.

Install dependencies:

sudo pacman -S cython
pip install --user pycocotools

Then run training and monitor the results:

./train.sh
tensorboard --logdir float:object_detection_models.float,quantized:object_detection_models.quantized,ppn:object_detection_models.ppn

Export model

Export the model for TensorFlow Lite:

./export.sh