On the Design of Privacy-Aware Cameras:
a Study on Deep Neural Networks

With the propagation of cameras in the public space, each year with higher definition, data privacy has become a general concern. We are in fact surrounded by surveillance systems and high resolution cameras that generate billions of images per day [bogdanchikov2019face]. These images can be processed using Computer Vision (CV) techniques in order to perform tasks such as object detection [he2017maskrcnn], semantic segmentation [chen2017deeplabv3], or object tracking [wang2019fast]. However, and most importantly, these same CV techniques can be used to extract privacy sensitive information, such as faces [schroff2015facenet] or license plates [chang2004automatic] recognition. The issue becomes even more problematic when these processes are performed without the notice or consent of consumers.

This situation raises many legal issues related to data protection. As a result, a great deal of countries have adopted strict legislation to tackle this problematic [unctad2020data]. For example, the European Union General Data Protection Regulation (GDPR) ensures confidentiality protection and transparency on the collection, storage and use of the data. However, there exist techniques for anonymizing sensitive information in visual tasks. Traditional approaches consist in blurring, masking or changing pixels corresponding to personal identifiers [hukkelaas2019deepprivacy], such as faces or license plate numbers. Sadly, although this type of solution is GDPR compliant, it is prone to error and there are no guaranties that sensitive data cannot be compromised. Additionally, usually owing to poor security, the cameras are not immune from being accessed directly by malicious people, thus putting in jeopardy the private personal data they record.

In view of these issues, Privacy by Design (PbD) seems to be a promising approach for the protection of private data [cavoukian2009privacy]. Its principle is to guarantee personal information protection since the very conception of the devices. In our particular case, when dealing with computer vision tasks, PbD can be achieved by limiting the visual quality with respect to the sensor capabilities. Indeed, by restricting the right portion of data acquired by the sensor, it is possible to highly limit the efficiency of privacy sensitive processing tasks while, in the same time, having low impact on the non sensitive ones.

In this work, we explore the feasibility of privacy-aware cameras that respect the PbD principle by studying their impacts on CV processing tasks which are usually applied to images taken in the public sphere. To do so, we simulate the behavior of such a privacy-aware camera with two main visual distortions. The first consists in generating defocus blur so that the images correspond to the outputs of a specially designed camera with fixed focal length, aperture and position of the in-focus plane; while the second distortion mimics a simple monochrome camera. Grayscale images may improve anonymisation as it reduces contextual information related to specific color characteristics of personal objects. We then study the effects of such distortions on Deep Neural Networks (DNN) applied to several CV tasks. These latter ones are classified into privacy-sensitive tasks, such as license plate recognition and non privacy-sensitive tasks, like object detection or semantic segmentation. Our goal is thus to find out if there is a level of distortion that allows to process images automatically in an efficient way while preserving the privacy of people.

The rest of this paper is organised as follows. Section 2 presents background information related to the existing literature. Section 3 shows how the chosen image distortions influence the performance of the CNNs, depending on the sensitivity to details that we relate to privacy information.

Recent works in Deep Learning explore the impact of visual deformations on the robustness of DNNs [dodge2016understanding, hendrycks2018benchmarking, wang2021data, zhou2017classification, rusak2020simple]. In [dodge2016understanding], Dodge and Karam study the influence of 5 different quality distortions on neural networks for image classification and conclude they are specially sensitive to blur and noise. Similarly, in [hendrycks2018benchmarking], Hendrycks and Dietterich create a new dataset with 15 transformations from Imagenet [deng2009imagenet], including Gaussian noise, defocus blur and fog. Then, they perform several tests on classification networks to explore their performance on out-of-distribution data and indicate which architectures are more robust to adversarial perturbations. Other works include similar experiments with different colorizations [de2021impact] and blur [zhou2017classification]. To overcome these limitations, some proposed approaches include these distortions in data augmentation to improve model efficiency [wang2021data, zhou2017classification, rusak2020simple]. Their results and considerations improve our understanding on the deep models and also allow to overcome some of the observed constraints. However, we may also use this intake to balance how much of information one can learn from the input images depending of the task to perform.

Previous works on data anonymisation can be divided in two branches: software-level and hardware-level anonymisation. The first group of methods are more commonly adopted as they consist on a pre-processing phase and do not demand a specific camera [besmer2009tagged, vishwamitra2017blur, vishwamitra2017towards, ishii2020privacy]. They consist in automatically filtering sensitive information (e.g., facial features, license plates) by blurring or masking them, while preserving realistic background. More refined works in face recognition, like [hukkelaas2019deepprivacy], propose to generate realistic faces to mask the original information while keeping data distribution. However, as mentioned in the last section, all of these methods are prone to error and if an attacker succeeds to access the code, the privacy of the method can be highly compromised [winkler2014security].

Very few works propose a hardware-level solution to this concern. This includes chip implementations with low-level processing arguably a type of software-level implementation [fernandez2014focal]. Also, some of them propose to explore low resolution sensors [ryoo2018extreme, chou2018privacy]. Others propose to explore defocus blur as an aberration related to the parameters of a well-designed sensor to preserve privacy [pittaluga2016pre]. However, this solution is model-based and does not provide any insight on the use of DNNs, as proposed in this work. More recently, in [hinojosa2021learning], Hijonosa et al. makes use of Deep Optics to design an optical encoder-decoder that produces a highly defocused image that still succeeds to perform pose estimation thanks to the jointly optimized optics. Yet, this solution is task-specific as the decoder is finetuned to fit the lens and Human Pose Estimation metrics.

In this work, we propose to study the impact of a synthetically defocused dataset on different tasks to ensure privacy and still perform well on general tasks. To this aim, as illustrated in 1, we adopt semantic segmentation for a non-privacy task and License Plate Detection and Recognition as a privacy task. In the following, we present background information on these domains.

In this section, we explain the steps followed to generate our datasets based on the Cityscapes one [Cordts2016Cityscapes]. The goal is, while synthetically mimicking the output of a real-world privacy-designed camera, to calibrate the camera in a manner to hinders LPDR while being robust to non privacy-sensitive tasks such as the semantic segmentation of urban areas.

To this aim, we propose two main experiments:

1. [label=()]

2. The first explores the effect of both defocus-blurred and “grayscaled” input images during training and inference, where we prove the potential of a deep network to extract useful contextual information despite the imposed visual aberrations (cf. Section 3.2);

3. The second compares the generalization capability of two deep neural models, trained with standard colored and all-in-focus images, on a privacy sensitive task and a non-privacy sensitive one (Section 3.3).

In this paper, we raise some privacy concerns related to the increasing exploitation of images in the public space. Hence, we discuss the principles of Privacy by Design, which stands that a product should respect consumers personal information since its very own conception. Finally, we propose an approach to perform hardware-level anonymization by using defocus blur and grayscale images as inputs to neural networks. We show through our experiments that it is possible to improve privacy without harming more general non sensitive tasks like semantic segmentation. The advantage of our module is that it presents a general approach which can also be extended to other CV tasks such as object detection or classification. To the best of our knowledge, our work is the first to propose coupling defocus blur and a DNN model to strategically hide identifiable information without compromising concurrent tasks of more general purposes. Future directions for experimenting on more efficient models of different tasks, increasing data augmentation and conceptualizing the co-design of a privacy-aware camera to perform tasks with DNNs.