Review of Generative Adversarial Networks in Image Generation

Wanle Chi; Yun Huoy Choo; Ong Sing Goh

doi:10.20965/jaciii.2022.p0003

single-jc.php

JACIII Vol.26 No.1 pp. 3-7

doi: 10.20965/jaciii.2022.p0003

(2022)

Review:

Views over last 60 days: 784

Review of Generative Adversarial Networks in Image Generation

Wanle Chi^*,,†, Yun Huoy Choo^, and Ong Sing Goh^**

^*Department of Information Technology, Wenzhou Polytechnic
University Town, Chashan, Wenzhou, Zhejiang 325035, China

^**Faculty of Information and Communication Technology, Universiti Teknikal Malaysia Melaka (UTeM)
Malacca 76100, Malaysia

^†Corresponding author

Received:

May 31, 2020

Accepted:

March 11, 2021

Published:

January 20, 2022

Keywords:

image generation, generative adversarial networks, machine learning, gradients disappearing, collapse mode

Abstract

Generative adversarial network (GAN) model generates and discriminates images using an adversarial competitive strategy to generate high-quality images. The implementation of GAN in different fields is helpful for generating samples that are not easy to obtain. Image generation can help machine learning to balance data and improve the accuracy of the classifier. This paper introduces the principles of the GAN model and analyzes the advantages and disadvantages of improving GANs. The applications of GANs in image generation are analyzed. Finally, the problems of GANs in image generation are summarized.

Review of GANs in image generation

Cite this article as:

W. Chi, Y. Choo, and O. Goh, “Review of Generative Adversarial Networks in Image Generation,” J. Adv. Comput. Intell. Intell. Inform., Vol.26 No.1, pp. 3-7, 2022.

Data files:

References

[1] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza et al., “Generative Adversarial Nets,” Proc. of the 27th Int. Conf. on Neural Information Processing Systems (NIPS’14), pp. 2672-2680, 2014.
[2] M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative adversarial networks,” Proc. of the 34th Int. Conf. on Machine Learning (ICML’17), pp. 214-223, 2017.
[3] I. Gulrajani, F. Ahmed, M. Arjovsky et al., “Improved training of Wasserstein GANs,” Proc. of the 31st Int. Conf. on Neural Information Processing Systems (NIPS’17), pp. 5769-5779, 2017.
[4] M. Mirza and S. Osindero, “Conditional Generative Adversarial Nets,” arXiv preprint, arXiv:1411.1784, 2014.
[5] T. Miyato and M. Koyama, “cGANs with Projection Discriminator,” 6th Int. Conf. on Learning Representations (ICLR2018), pp. 517-540, 2018.
[6] Y. Bengio, G. Mesnil, Y. Dauphin et al., “Better Mixing via Deep Representations,” Proc. of the 30th Int. Conf. on Machine Learning (ICML’13), pp. I-552-I-560, 2013.
[7] A. Radford, L. Metz, and S. Chintala, “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks,” arXiv preprint, arXiv:1511.06434, 2015.
[8] T. Salimans, I. Goodfellow, W. Zaremba et al., “Improved Techniques for Training GANs,” Proc. of the 30th Int. Conf. on Neural Information Processing Systems (NIPS’16), pp. 2234-2242, 2016.
[9] H. Zhang, I. Goodfellow, D. Metaxas et al., “Self-attention generative adversarial networks,” Proc. of the 36th Int. Conf. on Machine Learning, pp. 7354-7363, 2019.
[10] T. Miyato, T. Kataoka, M. Koyama et al., “Spectral Normalization for Generative Adversarial Networks,” 6th Int. Conf. on Learning Representations (ICLR2018), pp. 737-758, 2018.
[11] H. Zhang, T. Xu, H. Li et al., “Stack GAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks,” 2017 IEEE Int. Conf. on Computer Vision (ICCV), pp. 5908-5916, 2017.
[12] H. Zhang, T. Xu, H. Li et al., “ StackGAN++: Realistic image synthesis with stacked generative adversarial networks,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.41, pp. 1947-1962, 2018.
[13] R. Huang, S. Zhang, T. Li et al., “Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis,” 2017 IEEE Int. Conf. on Computer Vision (ICCV), pp. 5218-5236, 2017.
[14] G. Yildirim, N. Jetchev, R. Vollgraf et al., “Generating High-Resolution Fashion Model Images Wearing Custom Outfits,” 2019 IEEE/CVF Int. Conf. on Computer Vision Workshop (ICCVW), pp. 1783-1796, 2019.
[15] T. Zhang, M. Zhang, and P. Jiang, “Inter-frame video image generation based on spatial continuity generative adversarial networks,” Chinese High Technology Letters, Vol.28, pp. 843-851, 2018 (in Chinese).
[16] A. Brock, J. Donahue, and K. Simonyan, “Large Scale GAN Training for High Fidelity Natural Image Synthesis,” 7th Int. Conf. on Learning Representations (ICLR2019), pp. 2517-2540, 2019.
[17] S. Reed, Z. Akata, X. Yan et al., “Generative Adversarial Text to Image Synthesis,” Proc. of the 33rd Int. Conf. on Machine Learning, pp. 1060-1069, 2016.
[18] M. Zhu, P. Pan, and W. Chen, “DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis,” 2019 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 1221-1242, 2019.
[19] C. Li, K. Xu, J. Zhu et al., “Triple Generative Adversarial Nets,” Proc. of the 31st Int. Conf. on Neural Information Processing Systems (NIPS’17), pp. 1481-1496, 2017.
[20] Y. Ren, X. Yu, J. Chen et al., “Deep Image Spatial Transformation for Person Image Generation,” 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 7690-7699, 2020.
[21] W. Jiang, S. Liu, C. Gao et al., “PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer,” 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 5194-5202, 2020.
[22] D. Liu, C. Long, H. Zhang et al., “ArshadowGAN: Shadow generative adversarial network for augmented reality in single light scenes,” 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 8139-8148, 2020.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

[1] [1] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza et al., “Generative Adversarial Nets,” Proc. of the 27th Int. Conf. on Neural Information Processing Systems (NIPS’14), pp. 2672-2680, 2014.

[2] [2] M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative adversarial networks,” Proc. of the 34th Int. Conf. on Machine Learning (ICML’17), pp. 214-223, 2017.

[3] [3] I. Gulrajani, F. Ahmed, M. Arjovsky et al., “Improved training of Wasserstein GANs,” Proc. of the 31st Int. Conf. on Neural Information Processing Systems (NIPS’17), pp. 5769-5779, 2017.

[4] [4] M. Mirza and S. Osindero, “Conditional Generative Adversarial Nets,” arXiv preprint, arXiv:1411.1784, 2014.

[5] [5] T. Miyato and M. Koyama, “cGANs with Projection Discriminator,” 6th Int. Conf. on Learning Representations (ICLR2018), pp. 517-540, 2018.

[6] [6] Y. Bengio, G. Mesnil, Y. Dauphin et al., “Better Mixing via Deep Representations,” Proc. of the 30th Int. Conf. on Machine Learning (ICML’13), pp. I-552-I-560, 2013.

[7] [7] A. Radford, L. Metz, and S. Chintala, “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks,” arXiv preprint, arXiv:1511.06434, 2015.

[8] [8] T. Salimans, I. Goodfellow, W. Zaremba et al., “Improved Techniques for Training GANs,” Proc. of the 30th Int. Conf. on Neural Information Processing Systems (NIPS’16), pp. 2234-2242, 2016.

[9] [9] H. Zhang, I. Goodfellow, D. Metaxas et al., “Self-attention generative adversarial networks,” Proc. of the 36th Int. Conf. on Machine Learning, pp. 7354-7363, 2019.

[10] [10] T. Miyato, T. Kataoka, M. Koyama et al., “Spectral Normalization for Generative Adversarial Networks,” 6th Int. Conf. on Learning Representations (ICLR2018), pp. 737-758, 2018.

[11] [11] H. Zhang, T. Xu, H. Li et al., “Stack GAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks,” 2017 IEEE Int. Conf. on Computer Vision (ICCV), pp. 5908-5916, 2017.

[12] [12] H. Zhang, T. Xu, H. Li et al., “ StackGAN++: Realistic image synthesis with stacked generative adversarial networks,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.41, pp. 1947-1962, 2018.

[13] [13] R. Huang, S. Zhang, T. Li et al., “Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis,” 2017 IEEE Int. Conf. on Computer Vision (ICCV), pp. 5218-5236, 2017.

[14] [14] G. Yildirim, N. Jetchev, R. Vollgraf et al., “Generating High-Resolution Fashion Model Images Wearing Custom Outfits,” 2019 IEEE/CVF Int. Conf. on Computer Vision Workshop (ICCVW), pp. 1783-1796, 2019.

[15] [15] T. Zhang, M. Zhang, and P. Jiang, “Inter-frame video image generation based on spatial continuity generative adversarial networks,” Chinese High Technology Letters, Vol.28, pp. 843-851, 2018 (in Chinese).

[16] [16] A. Brock, J. Donahue, and K. Simonyan, “Large Scale GAN Training for High Fidelity Natural Image Synthesis,” 7th Int. Conf. on Learning Representations (ICLR2019), pp. 2517-2540, 2019.

[17] [17] S. Reed, Z. Akata, X. Yan et al., “Generative Adversarial Text to Image Synthesis,” Proc. of the 33rd Int. Conf. on Machine Learning, pp. 1060-1069, 2016.

[18] [18] M. Zhu, P. Pan, and W. Chen, “DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis,” 2019 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 1221-1242, 2019.

[19] [19] C. Li, K. Xu, J. Zhu et al., “Triple Generative Adversarial Nets,” Proc. of the 31st Int. Conf. on Neural Information Processing Systems (NIPS’17), pp. 1481-1496, 2017.

[20] [20] Y. Ren, X. Yu, J. Chen et al., “Deep Image Spatial Transformation for Person Image Generation,” 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 7690-7699, 2020.

[21] [21] W. Jiang, S. Liu, C. Gao et al., “PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer,” 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 5194-5202, 2020.

[22] [22] D. Liu, C. Long, H. Zhang et al., “ArshadowGAN: Shadow generative adversarial network for augmented reality in single light scenes,” 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 8139-8148, 2020.

Review of Generative Adversarial Networks in Image Generation

Wanle Chi*,**,†, Yun Huoy Choo**, and Ong Sing Goh**

Wanle Chi^*,,†, Yun Huoy Choo^, and Ong Sing Goh^**