A collection of metrics to evaluate layout generation that can be easily used in 🤗 huggingface evaluate.
- Install
evaluatelibrary
pip install evaluate- Load the layout metric and then compute the score
import evaluate
import numpy as np
# Load the evaluation metric named "creative-graphic-design/layout-alignment"
alignment_score = evaluate.load("creative-graphic-design/layout-alignment")
# `batch_bbox` is a tensor representing (batch_size, max_num_elements, coordinates)
# and `batch_mask` is a boolean tensor representing (batch_size, max_num_elements).
batch_bbox = np.random.rand(512, 25, 4)
# Note that padded fields will be set to `False`
batch_mask = np.full((512, 25), fill_value=True)
# Add the batch of bboxes and masks to the metric
alignment_score.add_batch(batch_bbox=batch_bbox, batch_mask=batch_mask)
# Perform the computation of the evaluation metric
alignment_score.compute()- Heusel, Martin, et al. "Gans trained by a two time-scale update rule converge to a local nash equilibrium." Advances in neural information processing systems 30 (2017).
- Li, Jianan, et al. "LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators." International Conference on Learning Representations. 2019.
- Lee, Hsin-Ying, et al. "Neural design network: Graphic layout generation with constraints." Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16. Springer International Publishing, 2020.
- Naeem, Muhammad Ferjad, et al. "Reliable fidelity and diversity metrics for generative models." International Conference on Machine Learning. PMLR, 2020.
- Li, Jianan, et al. "Attribute-conditioned layout gan for automatic graphic design." IEEE Transactions on Visualization and Computer Graphics 27.10 (2020): 4039-4048.
- Kikuchi, Kotaro, et al. "Constrained graphic layout generation via latent optimization." Proceedings of the 29th ACM International Conference on Multimedia. 2021.
- Arroyo, Diego Martin, Janis Postels, and Federico Tombari. "Variational transformer networks for layout generation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.
- Kong, Xiang, et al. "BLT: bidirectional layout transformer for controllable layout generation." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.
- Hsu, Hsiao Yuan, et al. "Posterlayout: A new benchmark and approach for content-aware visual-textual presentation layout." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.