FEATURED

The machine is taught beauty, or the Turing test for professional photographers / SurprizingFacts

The programmers from Google set out to instill the machine with artistic taste and began with aesthetically correct photo processing. The peculiarity of the concept of beauty is that it does not have clear criteria. The machine was to create these criteria. The training took place on the basis of a popular photo site, a contextual database with image metadata, and the subject of the treatment was panoramic spherical imagery from Google Street View. The resulting set of algorithms is called Creatism – a system of in-depth training for the creation of artistic content.

Creatism Creators Hui Fang and Meng Zhang believe that a beauty rating scale was developed based on the results that photographers can use in the future for objective comparisons. By results, they conducted what was called the Turing test for photographers. The researchers suggested that experts evaluate the mix of the best shots made by people and created by Creatism, not to mention that there are machines created in the set. 40% of artisanal intelligence were awarded "good pictures with artistic taste". The developers aim to help any amateur photographer turn his photo into a beautiful image without filters and settings. By pressing one button that launches Creatism.


Approaching the transmission of light to the natural and the most "deep", the algorithm sometimes made minor errors in gluing the panorama that can be seen in this photo.

The matrix of a digital camera is not capable of simultaneously recording information in dark areas of the image, for which more exposure is needed, and in light ones, where smaller exposures are sufficient. The dynamic range is the difference in the exposure levels between the darkest and lightest part of the image that can be reproduced without losing information. In completely black areas of the image (in overexposed), as well as in overexposed (underexposed) information can not be restored. Dark sections of the image can be highlighted, but already with distortions. With the HDR method, a picture of several photographs taken with different exposures is combined in a single 32-bit file.

A person's vision is capable of capturing a visual picture with a difference of 10-14 degrees of illumination under bright sunlight (in the sun the pupil does not have the ability to adapt To different illumination) and up to 24 steps in the dim light of stars (it is possible to adapt the pupil to light difference). We can see, but it's difficult to capture even a part of this range in the photo. The dynamic range of the usual negative film is about 9-11 stages of exposure, slide film – 5-6 steps, the matrix of the digital camera – from 8 to 11 for most digital cameras. Special cameras provide 17 or more shooting levels. It is also not easy to reproduce the real dynamic range. Photographic paper for example is capable of reproducing only 7-8 stages of exposure.

For the experiment were taken 15 000 professional photos from the site 500px.com with a resolution of at least 299 x 299 pixels. With their help, developers have taught Creatism to highlight the most interesting in the landscape. Then, based on 40,000 panoramic spherical landscapes in the national US, Canada and Europe, the algorithm was taught to work with color and light.

Then, each frame was amplified by a dynamic range, and applied its own "expressive filter" operation, improving shadows, lighting and color. To this end, traditionally for image processing, we used the model of a generative contested network – a model where one part of the program degrades the quality of the original, and another tries to restore it and learns "how to do it not."

The AVA database was used to create the final scoring scale, in which 250 000 images, and most importantly – a variety of metadata, including a large number of aesthetic estimates for each image, semantic tags of 60 Categories, as well as labels associated with the photographic style for professional sorting.

Based on the results of all processing operations, 400 photos of the experiment were mixed with 800 photos of AVA and evaluated by experts.

Photoexperts with vocational education and at least 2 years of experience were asked to assess what category the picture belongs to and give an appropriate assessment:

  1. Made a "soap box" – as it turned out, it happened. Neither settings nor focus.
  2. A good photo of an inexperienced beginner, but the artistic value is minimal.
  3. A semi-professional photo. Artistic taste is clearly visible.
  4. Made by a professional.

From the number of processed Creatism 40% of pictures fell into the category from 3 to 4 – that is, they were recognized as at least semi-professional. The average score was lower than 3.

The researchers published the best pictures processed by Creatism. Under each for comparison, a full panorama from which it was cut out is shown.