Embeddings remain the best way to represent image features but does not always capture all latent information.This is still a problem in representation learning with computer vision descriptors struggling with precision and accuracy. Improving image embedding with other features is necessary for tasks like image geolocation, especially for indoor scenes where descriptive cues can have less distinctive characteristics. We propose a model architecture that integrates image N-dominant colors, and color histogram vectors in different color spaces with image embedding from deep metric learning and classification perspectives. Our results indicate that the integration of color features improves image embedding, surpassing the performance of using embedding alone.
pip install -r requirements.txt
We use the Hotel-ID to Combat Human Trafficking 2022 - FGVC9
Prepare the dataset by running the randomHotelDataPrep2.ipynb
This should randomly select and create 1000 hotels from the original dataset from which the validation set is created too. To ensure consistency in image dimensions and reduce computational complexity, each image was resized to 256 by 256 pixels, you can experiment with these values.
To avoid any path error, make sure you have placed the downloaded dataset in the right directory and rename the folder if needed to meet the notebook code requiremnts.
This involves two key approaches: extracting N-dominant color palettes and computing color histograms.
- N-dominant color palette from each input image involves identifying a number of most prominent colors present in the image
- The color histogram captures the frequency distribution of colors in the image
We parameterize the number of dominant colors i.e N and the color space S such that N ∈ {5, 11, 18, 28, 43, 64, 100} and S ∈{RGB, HSV }
To achieve this, run the colorFeatures2.ipynb
To reproduce any part or section of our results, edit the corresponding python script to account for your experiment focus in the training loop and run the script. For example the train_dml.py can be used to reproduce all our results that use the deep metric learning.
To cite this work, please use:
@ARTICLE{10976713,
author={Bamigbade, Opeyemi and Scanlon, Mark and Sheppard, John},
journal={IEEE Access},
title={Improving Image Embeddings With Colour Features in Indoor Scene Geolocation},
year={2025},
volume={13},
number={},
pages={79860-79870},
keywords={Image color analysis;Geology;Feature extraction;Histograms;Computational modeling;Computer architecture;Vectors;Measurement;Image retrieval;Computer vision;Classification;color descriptor;deep metric learning;embeddings;image geolocation;image retrieval;indoor scenes},
doi={10.1109/ACCESS.2025.3564496}}